MTA System

The Owlat MTA (apps/mta/) is a custom Mail Transfer Agent that delivers emails via direct SMTP to recipient mail servers. It replaces third-party provider APIs with a self-hosted service that provides full control over sending reputation, ISP-specific throttling, IP warming, and bounce processing.

The MTA is the default email provider (EMAIL_PROVIDER=mta). SES and Resend are available as alternatives. See ADR-005 for the rationale behind building a custom MTA.

What the MTA Is

• A specialized delivery engine — optimized for getting emails into inboxes with maximum deliverability
• An intelligent rate controller — adaptive per-ISP throttling, IP warming, circuit breakers, engagement-based priority
• A bounce processor — dedicated SMTP server for DSN/ARF parsing with auto-suppression
• A reputation guardian — DNSBL monitoring, per-org circuit breakers, graceful degradation under pressure
• A stateless worker — delivers and reports back via webhooks; no long-term message storage

What the MTA Is Not

• Not a complete mail server — no message archival, no full-text search, no web UI for browsing messages (unlike Postal, Mailcow, or iRedMail)
• Not a standalone product — designed as a component of the Owlat platform, depends on the Convex backend for campaign orchestration, tracking, domain management, and billing
• Not a tracking system — click/open tracking happens upstream in the Convex backend, not in the MTA
• Not a spam filter — no inbound spam scoring; outbound content screening and attachment malware scanning are handled via the @owlat/email-scanner package (see Email Security)
• Not a user-facing service — API-only, managed programmatically; no admin dashboard or self-service UI

Design Philosophy

The MTA follows a depth over breadth approach: rather than building a complete mail server that does many things adequately, it focuses exclusively on outbound delivery intelligence. Every feature serves one goal — maximizing inbox placement while protecting sender reputation.

Key architectural decisions:

• Stateless process, stateful Redis — the MTA process itself holds no state; all intelligence data lives in Redis with TTLs. This enables horizontal scaling and zero-downtime deploys.
• Intelligence before delivery — every email passes through 10+ checks before touching SMTP. Most MTAs send first and react to failures; Owlat prevents failures proactively.
• Engagement-aware ordering — high-engagement recipients are sent first during IP warming, maximizing positive ISP signals when reputation matters most.
• Graceful degradation — back-pressure at the API level (429), per-domain backoff, emergency mode (503) when all IPs are blocked. The system protects itself and its customers automatically.

System Components

HTTP API Endpoints

The MTA supports two authentication modes: the master MTA_API_KEY (accepted on all endpoints) and per-organization API keys (accepted on /send endpoints only). Management endpoints require the master key.

Core

Credential Management

DKIM Management

Inbound Routing

Organization Limits

Pool Rules

Suppression List

Attachment Scanning

The scan endpoint performs two checks: file type validation (magic bytes, double extension, extension allowlist) and ClamAV malware scanning. The endpoint is fail-open — if ClamAV is unavailable, the file passes with a warning. See Email Security for details on the scanning pipeline.

Delivery Logs

Queue Inspection

Dead Letter Queue

Send Payload

interface EmailJob {
  messageId: string        // Unique ID for correlation
  to: string               // Recipient email
  from: string             // Sender address
  subject: string          // Email subject
  html: string             // HTML body
  text?: string            // Plain text body
  replyTo?: string         // Reply-to address
  headers?: Record<string, string>
  ipPool: 'transactional' | 'campaign'
  organizationId: string   // For circuit breaker scoping
  engagementScore?: number // 0-100 for priority ordering
  dkimDomain: string       // Domain for DKIM signing
}

Intelligence Pipeline

Every email passes through ten checks before SMTP delivery. If any check fails, the job is deferred (retried later) rather than dropped — except suppression and content screening, which silently drop the job.

0a. Content Pre-Screening

File: intelligence/contentScreening.ts

Pluggable content inspection step that runs before all other checks. Catches malformed or dangerous content before it can damage IP reputation. Enabled by default (CONTENT_SCREENING_ENABLED=true).

0b. Suppression List

File: intelligence/suppressionList.ts

Global suppression list checked before all other intelligence checks. If the recipient is suppressed, the job is silently dropped (no retry).

Suppressed addresses are stored in a Redis set, normalized to lowercase. The suppression list is managed via POST /suppression, DELETE /suppression/:email, and GET /suppression/check/:email.

1. Circuit Breaker

File: intelligence/circuitBreaker.ts

Per-organization bounce rate monitoring with three states:

Thresholds:

• Fast check (first 50 sends): trips if bounce rate > 15%
• Slow check (100+ sends): trips if bounce rate > 8%

When the circuit opens, a org.circuit_breaker webhook event is sent to Convex.

1b. Organization Rate Limits

File: intelligence/orgLimits.ts

Per-organization daily and hourly send rate limits enforced via Redis counters.

Per-org overrides can be set via POST /org-limits. When a limit is exceeded, the job is deferred with a retryAfter delay until the next period boundary (midnight UTC for daily, next hour for hourly).

2. Domain Throttle

File: intelligence/domainThrottle.ts

Adaptive per-domain rate limiting using a sliding window in Redis. Each ISP has a specific sending profile:

The throttle automatically:

• Backs off on SMTP 4xx responses (rate reduced by backoff factor)
• Recovers on sustained successful deliveries (rate increased by recovery factor)
• Blocks after repeated failures (auto-recovers after 5 minutes)

3. SMTP Response Tracking

File: intelligence/smtpResponse.ts

Tracks SMTP response codes (4xx/5xx) per recipient domain. Detects degraded and blocking conditions and suggests deferral delays based on response patterns.

3b. Domain Connection Backoff

File: scaling/degradation.ts

Tracks per-domain connection failures and applies automatic backoff on repeated failures. Jobs targeting domains in backoff are deferred with a calculated retry delay.

3c. Pool Rules Resolution

File: scaling/poolRules.ts

Resolves the effective IP pool and dedicated IP for the sending organization before IP selection. Resolution priority:

1. Organization-specific pool rule (if set via POST /pool-rules)
2. Request ipPool field from the job
3. Default pool

If a dedicated IP is configured for the organization, it bypasses round-robin selection.

4. IP Selection

File: scaling/ipPool.ts

Round-robin IP selection within the resolved pool. IPs flagged by DNSBL checking are excluded. If a dedicated IP was resolved in step 3c, it is used directly. If all IPs in a pool are blocked, the system falls back to the first IP and sends an all_ips_blocked alert.

4b. Warming Cap

File: intelligence/warming.ts

New IPs follow a 30-day warming schedule with daily send caps that grow gradually:

The warming system adapts based on deliverability:

• Accelerates when bounce and deferral rates are low
• Decelerates when deliverability signals are poor

Daily stats are tracked in Redis. When an IP graduates, an ip.warming_complete webhook event is sent.

5. DNSBL Checking

File: intelligence/dnsbl.ts

Periodic check (every 15 minutes) against DNS-based blocklists:

Listed IPs are automatically removed from the active sending pool. Convex is notified via ip.blocklisted and ip.delisted webhook events.

6. Engagement Priority

File: intelligence/engagementPriority.ts

Maps engagement scores (0–100, provided by the Convex backend) to delivery priority. High-engagement contacts (score 80+) are processed first using GroupMQ's orderMs mechanism.

Queue System

Files: queue/setup.ts, queue/handler.ts, queue/groups.ts

The MTA uses GroupMQ, a Redis-backed job queue with group processing.

Group Key

Jobs are grouped by {ipPool}:{recipientDomain} — for example, campaign:gmail.com. This ensures that emails to the same ISP from the same IP pool are processed sequentially, respecting per-domain rate limits. Different domains process in parallel.

Worker Configuration

When an intelligence check defers a job, it uses GroupMQ's DeferError mechanism to reschedule with a specific delay.

SMTP Delivery

MX Resolution

File: smtp/mxResolver.ts

DNS MX lookups with Redis caching. Each recipient domain's MX records are resolved and tried in priority order. If all MX hosts fail, the domain is recorded in the degradation system.

DKIM Signing

Files: smtp/dkimStore.ts, smtp/dkim.ts

All outbound mail is signed with DKIM. Keys are stored in Redis with a 5-minute in-memory cache for performance. On startup, keys from the DKIM_KEYS environment variable are seeded into Redis (existing Redis keys are not overwritten).

Key management:

• API management — POST /dkim, GET /dkim, DELETE /dkim/:domain
• Key rotation — POST /dkim/:domain/rotate generates a new RSA 2048-bit key pair and returns the DNS TXT record value (v=DKIM1; k=rsa; p={base64}) for publishing
• Env var seeding — DKIM_KEYS still works for initial bootstrapping; Redis is the source of truth at runtime

Connection Pool

File: smtp/connectionPool.ts

Reusable Nodemailer transport pool keyed by {mxHost}:{bindIp}:{dkimDomain}. Replaces single-use transports for better connection reuse and reduced DNS/TLS overhead.

The pool runs a periodic eviction sweep (every 10s) to close idle and aged-out transports. Connections with in-flight sends are never evicted. A Prometheus gauge (mta_smtp_pool_connections) tracks active and idle connection counts by pool key.

On shutdown, the pool drains all in-flight sends before closing transports.

VERP Return-Path

File: bounce/verp.ts

Variable Envelope Return Path encoding embeds the original message ID into the bounce address:

bounces+{messageId}@bounces.owlat.com

This allows the bounce processor to correlate incoming DSN messages back to the original send without maintaining a lookup table.

Transport

Nodemailer transports are acquired from the connection pool for each send:

• Binds to a specific IP from the selected pool (or dedicated IP)
• 30-second connection timeout, 60-second socket timeout
• No TLS requirement (port 25 MX delivery)
• DKIM signing keys loaded from the DKIM key store

SMTP Submission Server

File: smtp/submissionServer.ts

The MTA includes an optional SMTP submission server for traditional email client compatibility. Disabled by default.

Authentication: Accepts both the master MTA_API_KEY and per-org API keys as the SMTP password (username is ignored). Per-org keys scope all submitted emails to that organization.

Processing: Incoming messages are parsed with mailparser, then fanned out to one GroupMQ job per recipient (To, Cc, Bcc). Each job enters the standard intelligence pipeline. The sender domain is extracted for DKIM signing, and each job is assigned a smtp-{uuid} message ID.

Bounce Processing

Files: bounce/server.ts, bounce/parser.ts, bounce/fblProcessor.ts, bounce/classifier.ts

Inbound SMTP Server

A dedicated SMTP server listens on the bounce port (default: 25) for incoming delivery status notifications and feedback loop reports. Messages arrive at the VERP return-path address.

Processing Pipeline

1. VERP decode — extract the original message ID from the recipient address
2. DSN parse — parse RFC 3464 delivery status notifications for enhanced status codes and diagnostic information
3. ARF/FBL parse — detect Abuse Reporting Format feedback loop reports from ISPs
4. Classify — categorize the event:
   • Hard bounce (permanent) — invalid recipient, domain doesn't exist
   • Soft bounce (temporary) — mailbox full, server temporarily unavailable
   • Complaint — recipient marked the email as spam
5. Auto-suppress — hard bounces and complaints automatically add the recipient to the suppression list
6. Webhook — post the event to the Convex backend for tracking and contact suppression

Per-Organization Credentials

File: auth/credentials.ts

API keys formatted as owlat_{32-char-hex} provide per-organization isolation. Each credential is stored in Redis and indexed by organization ID.

• The master MTA_API_KEY continues to work for all endpoints (used by the Convex backend)
• Per-org keys authenticate both HTTP API /send requests and SMTP submission connections
• Management endpoints (POST/GET/DELETE /credentials) require the master key

Credentials track createdAt and lastUsedAt timestamps. The GET /credentials endpoint returns truncated key prefixes (first 10 chars) for safe display.

Inbound Email Routing

Files: inbound/router.ts, inbound/forwarder.ts

Route incoming emails by domain and local-part address. Routes are stored in Redis and managed via the /inbound/routes API.

Route Modes

Route Matching

Priority: exact address match, then wildcard (*). For example, a route for support@example.com takes precedence over *@example.com. Routes are keyed by domain:address in Redis.

Webhook Forwarding

For endpoint mode, the parsed email is posted as JSON to the configured URL:

• Payload includes: from, to, subject, textBody, htmlBody, headers, date, messageId, inReplyTo, references, and attachments (base64-encoded)
• 10-second timeout per request
• Up to 2 retries with exponential backoff (1s, 2s)

An inbound.received webhook event is sent to Convex for all routed inbound emails.

IP Pool Management

Files: scaling/ipPool.ts, scaling/degradation.ts

Two Pools

IPs are configured via IP_POOLS_TRANSACTIONAL and IP_POOLS_CAMPAIGN environment variables (comma-separated).

Selection

Round-robin IP selection within each pool. IPs flagged by DNSBL checking are excluded from selection. If all IPs in a pool are blocked, the system falls back to the first IP and sends an all_ips_blocked alert.

Pool Rules

File: scaling/poolRules.ts

Per-organization IP pool assignment rules stored in Redis. A rule can override the pool or assign a dedicated IP that bypasses round-robin selection.

Resolution order:

1. Organization-specific rule (if set via POST /pool-rules)
2. Request ipPool field from the job
3. Default pool

Managed via POST /pool-rules, GET /pool-rules/:orgId, DELETE /pool-rules/:orgId.

Degradation

The degradation system tracks per-domain connection failures and applies automatic backoff on repeated failures. System-level health checks monitor queue depth and Redis connectivity. When the queue is too deep, the API returns HTTP 429 (backpressure).

Webhook Events

Events are posted to {CONVEX_SITE_URL}/webhooks/mta with authentication via the X-MTA-Secret header (constant-time comparison) and timestamp validation (5-minute tolerance).

Convex Integration

The Convex backend receives MTA webhooks via mtaWebhook.ts (HTTP action). Bounce and complaint events reuse the existing processBounceEvent and processComplaintEvent mutations, so they feed into the same contact suppression and analytics pipeline used by SES/Resend webhooks.

The MtaProvider class in lib/emailProviders/mta.ts implements the EmailProvider interface, posting send requests to the MTA's HTTP API with retry logic for transient errors (429, 5xx) and a 30-second request timeout.

Monitoring

Prometheus Metrics

File: monitoring/collector.ts

Exposed at GET /metrics. Key metrics include:

• Emails sent/failed/deferred (by domain, pool)
• Queue depth and processing latency
• Circuit breaker state changes
• DNSBL check results
• Warming progress per IP
• SMTP connection pool size (active/idle)

Structured Logging

File: monitoring/logger.ts

Pino JSON logging with configurable log level (LOG_LEVEL env var). All log entries include structured context (message ID, domain, IP pool, organization ID).

Configuration

All configuration is loaded from environment variables via config.ts.

Required Variables

Optional Variables

ClamAV Sidecar (Docker)

The MTA can optionally run ClamAV as a Docker sidecar for attachment malware scanning. ClamAV provides the clamd daemon on port 3310, with freshclam auto-updating virus definitions inside the official image.

# docker-compose.yml (excerpt)
clamav:
  image: clamav/clamav:1.3
  ports:
    - '3310:3310'
  volumes:
    - clamav-data:/var/lib/clamav
  healthcheck:
    test: ["CMD", "clamdcheck"]
    interval: 60s
    timeout: 10s
    retries: 3

The @owlat/email-scanner package provides a TCP client for the clamd INSTREAM protocol. The MTA's /scan/attachment endpoint combines file type validation with ClamAV scanning.

Startup Sequence

1. Load configuration from environment variables
2. Set organization rate limit defaults
3. Configure SMTP connection pool and start eviction timer
4. Connect to Redis and verify connectivity
5. Seed DKIM keys from DKIM_KEYS env var into Redis (existing keys not overwritten)
6. Initialize IP pools in Redis (mark all IPs as active)
7. Initialize warming state for each IP
8. Create GroupMQ queue and worker
9. Start HTTP server (Hono on PORT)
10. Start bounce SMTP server (on BOUNCE_PORT)
11. Start SMTP submission server (on SUBMISSION_PORT, if SUBMISSION_ENABLED=true)
12. Start DNSBL checker (runs every 15 minutes)
13. Start warming evaluation cron (runs every hour)
14. Start GroupMQ worker (begins processing jobs)

Graceful Shutdown

On SIGTERM/SIGINT:

1. Stop periodic crons (DNSBL checker, warming evaluation)
2. Close the HTTP server
3. Close the bounce SMTP server
4. Close the SMTP submission server (if running)
5. Drain the GroupMQ worker (wait for in-flight jobs to complete)
6. Drain and close the SMTP connection pool
7. Close the Redis connection

Key Files