Postbox Architecture

Postbox is Owlat's personal-mail feature: per-user mailboxes with a webmail UI and native IMAP4rev1 / SMTP submission support. This page covers the implementation. For the user-facing guide see Postbox in the product guide.

Component map

                        ┌────────────────────────────────────┐
   ┌──────────────────► │ apps/api (Convex)                  │ ◄── webmail UI
   │                    │  ├── mail/mailbox / mail/imap      │     /dashboard/postbox/*
   │                    │  ├── mail/folders / mail/labels    │
   │                    │  ├── mail/drafts + draftLifecycle  │
   │                    │  ├── mail/outbound (+ lifecycle)   │
   │                    │  ├── mail/filters / mail/aliases   │
   │                    │  ├── mail/appPasswords (PBKDF2)    │
   │                    │  ├── mail/authHttp (HMAC verify)   │
   │                    │  └── mail/delivery (inbound route) │
   │                    └────────────────────────────────────┘
   │                              ▲              ▲
   │                              │ webhook      │ verify-cred
   │                              │ (delivery)   │ (HMAC)
   │                              │              │
┌──┴───────────┐     SMTP    ┌────┴──────────────┴────┐    IMAP/SMTP    ┌──────────────┐
│ apps/web     │ submission  │ apps/mta               │    submission   │ native       │
│ (composer)   │────────────►│  outbound queue        │◄────────────────│ clients      │
└──────────────┘             │  inbound routing       │                 │ (Apple Mail, │
                             └────────┬───────────────┘                 │  Thunderbird,│
                                      │  bind to                        │  mobile)     │
                                      │  mailboxResolver                └──────┬───────┘
                                      ▼                                        │
                             ┌────────────────────────┐                        │
                             │ apps/imap              │ ◄──────────────────────┘
                             │  IMAP4rev1 server      │     IMAP fetch / store
                             │  port 993 implicit TLS │
                             └────────────────────────┘
                                      │
                                      └─── reads/writes via Convex client
                                           (uses CONVEX_URL + CONVEX_ADMIN_KEY)

Schema

The mail tables are defined in apps/api/convex/schema/mail.ts (re-exported into schema.ts as mailTables and spread into defineSchema()). Key relationships:

mailboxes (1)──(*) mailFolders (*)──(*) mailMessages (*)──(1) mailThreads
         ├────(*) mailAliases
         ├────(*) mailAppPasswords
         ├────(*) mailFilters
         ├────(*) mailLabels
         ├────(*) mailDrafts
         ├────(*) mailSignatures
         ├────(*) mailForwarding
         ├────(*) mailVacationResponders / mailVacationLog
         ├────(*) mailContacts
         └────(*) mailAuditLog / mailAuthFailures

externalMailAccounts (1)──(1) mailboxes (kind='external')
                     └────(*) externalMailFolderSync
pendingMailboxes        — reserved-mailbox intent attached to an invitation

There is no mailOutbound table — outbound state lives in the embedded mailMessages.outbound object (a denormalized aggregate plus a per-recipient array). There is no mailIdentities table — sending identities are a module (mail/identities.ts) layered over mailSignatures and the mailbox's allowed-from set. There is no mailSnooze table — snooze is the snoozedUntil / snoozedFromFolderId fields on mailMessages, swept by a 1-minute cron via the by_snoozed_until index.

Index notes:

• mailMessages inbox sorting uses by_mailbox_and_received on [mailboxId, receivedAt]. IMAP UID ranges use by_folder_and_uid on [folderId, uid]; CONDSTORE fast-resync uses by_folder_and_modseq. Thread reads use by_thread; the snooze cron uses by_snoozed_until. There is also a search_messages full-text search index on snippet. (There is no internalDate index.)
• mailAppPasswords stores PBKDF2-SHA256 hashes (100k iterations, encoded <salt-hex>:<hash-hex>) — the cleartext is shown to the user exactly once on creation. A separate passwordPrefix (first 4 chars) narrows the candidate set via by_prefix before the deliberately slow hash compare.
• mailAliases stores the canonical lowercase address in the alias field and is indexed by by_alias on [alias] (plus by_target). organizationId is stored on the row but is not part of the lookup index — the inbound router resolves a recipient by the alias field alone.

Module layout (Convex)

All mail modules live under apps/api/convex/mail/.

App-password auth flow

Native IMAP/SMTP clients can't use the dashboard session. They authenticate with app passwords — single-mailbox, revocable tokens stored as PBKDF2-SHA256 hashes in mailAppPasswords. Both verification paths terminate in the same internal Convex action, internal.mail.appPasswords.verify, but they reach it differently.

IMAP (apps/imap) — the IMAP server holds CONVEX_ADMIN_KEY and calls the action over the Convex client directly:

1.  user opens /dashboard/postbox/settings/app-passwords
2.  mail/appPasswords.generate creates a row with PBKDF2(password); returns plaintext once
3.  user pastes plaintext into Apple Mail; client connects to apps/imap (port 993)
4.  commands/login calls the Convex action `mail/appPasswords:verify`
        ({ address, password }) over the admin-key client
5.  appPasswords.verify narrows candidates by passwordPrefix, runs PBKDF2(password),
        and returns { ok: true, mailboxId, appPasswordId, ... } on success
6.  apps/imap binds the IMAP session to that mailboxId
        — every subsequent IMAP command runs against Convex as that mailbox

SMTP submission (via apps/mta) — the MTA does not hold the admin key, so it posts to the HMAC-signed HTTP endpoint instead:

1.  desktop client submits over SMTP to apps/mta's submission port
2.  apps/mta POSTs /webhooks/mta-verify-credential
        body:    { address, password, scope: 'imap' | 'smtp' }
        headers: x-mta-signature: HMAC-SHA256(`${timestamp}.${body}`, MTA_WEBHOOK_SECRET)
                 x-mta-timestamp: <unix-seconds>
3.  mail/authHttp.ts verifies the HMAC + freshness, then delegates to
        internal.mail.appPasswords.verify, returning { ok, mailboxId, appPasswordId, ... }

The HMAC pattern means the MTA never needs the Convex admin key for credential checks — only the shared MTA_WEBHOOK_SECRET. The IMAP server, which runs the full command loop against Convex, does hold CONVEX_ADMIN_KEY.

IMAP server (apps/imap)

Configuration is read from env on startup:

Source layout:

apps/imap/src/
├── index.ts        # process entry, signal handling
├── server.ts       # net.createServer + TLS upgrade
├── connection.ts   # IMAP pump: socket lifecycle, line/literal buffering
├── parser.ts       # IMAP command parser
├── mime.ts         # RFC 5322 / 2045 parser
├── convex.ts       # Convex client wrapper used by command modules
├── rateLimit.ts    # per-IP and per-credential throttling
├── config.ts       # env loading
├── logger.ts
└── commands/       # one module per IMAP verb (ADR-0016)
    ├── walker.ts   # typed dispatch registry + CAPABILITY-line assembly
    ├── types.ts    # ImapVerb, CommandModule, session/deps types
    ├── helpers/    # shared session helpers
    └── <verb>/index.ts   # login, select, fetch, store, copy, move, idle, …

connection.ts is the pump — it owns the socket, line buffering, and literal absorption, and knows nothing about IMAP verbs. Command handling was extracted into per-verb modules under commands/<verb>/index.ts, dispatched through commands/walker.ts (see ADR-0016 and the header comment in connection.ts). Multi-verb modules (LIST + LSUB, SELECT + EXAMINE, UNSELECT + CLOSE) register themselves under each verb they declare; the walker also assembles the CAPABILITY line from each module's declared atoms.

Each connection lifecycle: * OK greeting → LOGIN/AUTHENTICATE (→ mail/authHttp) → SELECT inbox → command loop (FETCH, STORE, COPY, MOVE, IDLE, …).

Outbound: composing → SMTP

Sending mail from Postbox goes through the existing MTA, not the IMAP server. The flow is:

1. User clicks Send. The web app moves the draft into pending_send (undo-send window) and schedules internal.mail.outbound.dispatchDraft (apps/api/convex/mail/outbound.ts).
2. dispatchDraft (a Node action) validates the draft state and undo token, scans each attachment through the MTA's ClamAV endpoint (fail-open on outage), renders the final HTML + plain-text bodies through @owlat/email-renderer, builds an RFC 5322 multipart message, and stores the raw .eml in ctx.storage. It then hands off to internal.mail.draftLifecycle.transition({ to: 'sent' }), which atomically inserts the Sent-folder mailMessages row with outbound.state='queued' and deletes the draft. See ADR-0028.
3. For a hosted mailbox it POSTs one MTA /send per recipient, prefixing the MTA message id with pb-<mailMessageId>-<idx> so the bounce/sent webhook can look the row back up. A synchronous 5xx transitions that recipient to bounced; a network error transitions it to failed — both via internal.mail.postboxOutboundLifecycle.transition. (For an external mailbox it takes the single-POST dispatchViaExternalWorker path described below.)
4. Asynchronous MTA delivery events arrive at POST /webhooks/mta. The webhook ceremony (rate limit, signature check, audit, parse, dispatch) lives in webhooks/pipeline.ts + webhooks/adapters/mta.ts; mtaWebhook.ts is just a thin httpAction entry that delegates to runInboundPipeline. For events whose provider message id has the pb- prefix, the dispatcher (webhooks/dispatcher.ts) calls internal.mail.postboxOutboundLifecycle.transitionByMtaMessageId, which parses the id and applies the per-recipient transition.

postboxOutboundLifecycle.ts is the sole writer of every mailMessages.outbound.recipients[].state and the only producer of the derived mailMessages.outbound.state aggregate (see ADR-0012). The per-recipient states are queued | sent | bounced | failed; bounced and failed are terminal. The aggregate column adds one extra literal, partial, when the recipients are in a mix of states. (Personal mail discards the hard/soft bounce classification — that is a campaign-side concern.)

Native SMTP submission from a desktop client takes the same path: the MTA accepts SMTP on its submission port, calls mail/authHttp to verify the credential against mailAppPasswords, then enqueues the message normally.

Inbound: MX → mailbox

inbound SMTP ──► apps/mta ──► mailboxResolver (apps/mta/src/inbound)
                                  │
                                  │ POST /webhooks/mta-mailbox
                                  ▼
                         mail/webhook.ts ──► mail/delivery.ts
                                              │
                                              ├── resolve alias → mailbox
                                              ├── apply mailFilters
                                              ├── insert mailMessages row
                                              ├── run mail/deliveryHooks
                                              └── notify subscribers (Convex realtime)

The MTA routes personal-mailbox deliveries to the dedicated POST /webhooks/mta-mailbox route (apps/api/convex/http.ts, handled by handleMailWebhook from mail/webhook.ts); the routing decision is in apps/mta/src/webhooks/convexNotifier.ts (inbound.mailbox.received → /webhooks/mta-mailbox, everything else → /webhooks/mta).

mailAliases is the lookup table — every accepted recipient address must match an alias row before delivery. Spam and ClamAV checks (when enabled) run before insert.

External mailboxes (mail.external)

Separate from hosted Postbox, the mail.external flag (label "Connect external mailbox") lets each user connect their own existing Gmail / Fastmail / company mailbox over IMAP+SMTP — personal mail without registering a sending domain. It is independent of the hosted postbox flag (intentionally not requires: ['postbox']), activates the external-mail Docker Compose profile, and runs the apps/mail-sync worker.

How it differs from a hosted mailbox:

• A connected account creates a mailboxes row with kind='external' linked 1:1 to an externalMailAccounts row (host/port/TLS, auth method, encrypted credential envelope, connection status).
• The worker is the client of the remote IMAP server. externalMailFolderSync tracks per-(account, folder) incremental fetch cursors (remoteUidValidity, lastSeenUid, optional CONDSTORE lastSeenModseq), distinct from mailFolders' own UID state (which tracks Owlat-as-IMAP-server).
• Inbound messages fetched from the remote server are ingested via mail/externalDelivery.ts (ingestExternalRaw / ingestExternalMessage).
• Outbound for an external mailbox skips the per-recipient MTA path: dispatchDraft checks internal.mail.externalAccounts.resolveOutboundTransport, and for kind='external' calls dispatchViaExternalWorker — a single POST to the worker's /send, which sends through the user's own SMTP and APPENDs the sent copy to the remote Sent folder. SMTP is synchronous, so the worker's per-recipient result maps straight onto postboxOutboundLifecycle with no webhook.

The connect/test/credential modules are externalAccounts.ts (queries + internal mutations), externalAccountsActions.ts (connect / testConnection / credential handling), and externalDelivery.ts (ingest).

Integration points

Where to start when extending

• New folder type / system folder → mail/folders.ts + dashboard sidebar in apps/web/app/components/postbox/
• New filter action → mail/filters.ts rule executor + UI in /dashboard/postbox/settings/filters
• New IMAP command → add a apps/imap/src/commands/<verb>/index.ts module and register it in commands/walker.ts (see ADR-0016)
• New outbound state transition → extend mail/postboxOutboundLifecycle.ts (the single writer of outbound state)
• Server-side enforcement of the feature flag → check isFlagEnabled(stored, 'postbox') (or 'mail.external') early in every mutation