Providers

Wherever Owlat talks to an external system — an LLM, an email delivery backend, a notification transport, a vector store, an analytics sink — it goes through a provider abstraction. Most are factories that read a single env var to pick an implementation, cache it per-process, and expose a small, stable interface to the rest of the codebase.

The point: self-hosters swap providers without code changes, and we can add new ones without touching every call site.

Two pluggable provider abstractions ship: the LLM provider and the send (email) provider, both fully consumed by the backend (agent steps, knowledge extraction, translation, every send path). The LLM provider reads one env var (LLM_PROVIDER) and caches the resolved client per-process. The send (email) provider is the exception: it uses a static registry keyed by provider kind (passed in per call) instead of reading one env var and caching the resolved instance. That difference is called out below.

Factory layout

The two shipping abstractions live under apps/api/convex/lib/:

lib/
├── llmProvider.ts          ← LLM_PROVIDER — single module
│                              exports getLLMProvider(task) / getEmbeddingModel() / getLLMConfig()
├── sendProviders/          ← registry keyed by SendProviderKind (not one env var)
│   ├── types.ts            ← SendProviderModule<K>, EmailSendAttempt, EmailErrorCode
│   ├── index.ts            ← SEND_PROVIDERS registry + providerFor(kind)
│   ├── dispatch.ts         ← sendProviderDispatch() — owns the retry loop
│   ├── routing.ts          ← resolveRoute() — per-org route selection
│   ├── health.ts           ← providerHealth recording + reads
│   ├── mta/index.ts        ← built-in MTA adapter
│   ├── ses/index.ts        ← Amazon SES adapter
│   ├── resend/index.ts     ← Resend adapter
│   └── strategies/         ← single / priority_failover / workload_split
└── emailProviders/         ← identity & domain verification only (NOT the send factory)
    ├── domainVerification.ts
    ├── mtaIdentity.ts
    └── sesIdentity.ts

LLM provider

Env var: LLM_PROVIDER (default: openai)

Supported values: openai (default), openrouter, ollama

All three speak the OpenAI Chat Completions shape (so anything OpenAI-compatible plugs in — a self-hosted vLLM or LM Studio, for example). Every client is built with createOpenAI from the Vercel AI SDK.

The whole abstraction is a single module, apps/api/convex/lib/llmProvider.ts, exposing standalone functions:

// apps/api/convex/lib/llmProvider.ts
export function getLLMProvider(task: LLMTask): LanguageModel;   // returns AI SDK LanguageModel
export function getEmbeddingModel(): EmbeddingModel;
export function getLLMConfig(): {
  provider: string;
  modelFast: string;
  modelCapable: string;
  embeddingModel: string;
  baseURL: string | undefined;
  hasApiKey: boolean;        // snapshot — never the key itself — safe to log
};

// task tiers — classify/extract/guard/summarize → fast model
//              draft/plan                       → capable model

To run Claude models, point the OpenAI client at an OpenAI-compatible endpoint — there is no native Anthropic provider, and anthropic is not a recognized LLM_PROVIDER value:

LLM_PROVIDER=openai
LLM_BASE_URL=https://api.anthropic.com/v1/   # or an OpenAI-compat shim
LLM_MODEL_CAPABLE=claude-sonnet-4-6
LLM_MODEL_FAST=claude-haiku-4-5-20251001

Required env vars depend on which provider is selected:

All OpenAI-compatible providers accept any of LLM_API_KEY, OPENROUTER_API_KEY, or OPENAI_API_KEY (first set wins — see resolveApiKey() in apps/api/convex/lib/llmProvider.ts). The ollama default base URL is the Docker service hostname http://ollama:11434/v1, resolved in resolveBaseURL() in the same file.

The ai feature flag requires LLM_PROVIDER and LLM_API_KEY to be set. The admin UI blocks toggling ai on until they exist.

Send (email) provider

The send-side abstraction lives under apps/api/convex/lib/sendProviders/ (per ADR-0020). Unlike the other factories it is not selected by reading one env var into a cached singleton — instead it is a static registry keyed by provider kind, and the kind is resolved per send (from the org's route config, falling back to the EMAIL_PROVIDER env var, then to unconfigured when neither names a provider). Resolution is fail-closed: there is no implicit MTA default, so a send that reaches dispatch without a configured provider is refused rather than dispatched to a phantom MTA. The send entry points gate on an isDeliveryConfigured capability check first (so a campaign or transactional send is rejected before any row is written), making this routing fallback defence-in-depth.

Supported kinds: mta, resend, ses

mta is the built-in custom sender that ships with self-host (apps/mta). resend and ses are alternatives — useful if you don't want to run an MTA, or for transactional traffic specifically while keeping campaigns on the MTA.

Registry & lookup

sendProviders/index.ts exports the registry and the lookup helpers — there is no getEmailProvider():

export const SEND_PROVIDERS = {
  mta: mtaSendProvider,
  ses: sesSendProvider,
  resend: resendSendProvider,
} as const;

export function providerFor<K extends SendProviderKind>(kind: K): SendProviderModule<K>;
export function isSendProviderKind(kind: string | undefined | null): kind is SendProviderKind;

A compile-time satisfies-style mapped-type check pins each registry value to SendProviderModule<thatKind>, so a missing method fails the build.

Adapter interface

Each adapter implements SendProviderModule<K> from sendProviders/types.ts:

export interface SendProviderModule<K extends SendProviderKind> {
  readonly kind: K;
  readonly retryDelays: readonly number[];                 // backoff schedule; dispatch owns the loop
  sendEmail(params: EmailSendParams, extras?: ExtrasFor<K>): Promise<EmailSendAttempt>;
  categorizeError(message: string, httpStatus?: number): EmailErrorCode;
}

// Single-attempt result — no internal retry:
export type EmailSendAttempt =
  | { success: true; id: string }
  | { success: false; errorMessage: string; errorCode: EmailErrorCode };

sendEmail performs exactly one attempt and never retries internally. extras is a typed per-provider second arg (MtaExtras carries ipPool/engagementScore/dkimDomain; SES and Resend take none). EmailErrorCode is an enum (RATE_LIMIT, SERVER_ERROR, INVALID_RECIPIENT, INVALID_SENDER, AUTH_FAILED, CONTENT_REJECTED, UNKNOWN); only RATE_LIMIT and SERVER_ERROR are retryable.

Dispatch

The single public entry point is sendProviderDispatch() in sendProviders/dispatch.ts:

sendProviderDispatch(ctx, kind, params, extras?): Promise<DispatchResult>

It owns the retry loop (driven by the module's retryDelays + categorizeError), and after every terminal outcome — success or exhausted retries — it records provider health by scheduling internal.lib.sendProviders.health.recordSendResult. DispatchResult carries the final EmailSendAttempt, the providerType used, total latencyMs, and the number of attempts.

Routing & health

resolveRoute() in sendProviders/routing.ts picks the provider kind for a send from an org's providerRoutes config, dispatching to a strategy module under sendProviders/strategies/ — single, priority_failover, or workload_split. It falls back to the EMAIL_PROVIDER env var, then returns null (unconfigured) — never a phantom mta — when there is no config, no enabled providers, the strategy returns null, and EMAIL_PROVIDER names no provider.

Admins configure these routes per message type (transactional, campaigns, automations) under Settings → Technical → Provider Routing in the dashboard — pick the strategy, order the providers, set workload-split weights, and optionally pin an IP pool. With no route configured, every send uses the EMAIL_PROVIDER value; with neither a route nor EMAIL_PROVIDER, the send is refused (no delivery provider configured). See Operating Modes for which deployment shapes need a provider.

sendProviders/health.ts records rolling success/failure counts, average latency, and a healthy/degraded/down status per provider into the providerHealth table. Dispatch is the only writer; resolveRoute is the only reader of the all-providers snapshot.

Identity & domain verification

The older lib/emailProviders/ directory still exists, but it now holds only identity and domain-verification helpers — domainVerification.ts, mtaIdentity.ts, sesIdentity.ts — not the send factory.

See Environment Variables → Email Sending for the per-provider env vars.

Adding a new provider

LLM: because every supported client speaks the OpenAI shape, most "new providers" need no code at all — set LLM_PROVIDER=openai and point LLM_BASE_URL at the OpenAI-compatible endpoint. For a genuinely non-OpenAI-compatible backend you would edit the single apps/api/convex/lib/llmProvider.ts module directly (e.g. add a base-URL case in resolveBaseURL() and, if it needs a different SDK client, branch in getClient()). If the provider needs new env vars, declare them in FEATURE_FLAGS[<flag>].requiredEnvVars in packages/shared/src/featureFlags.ts so the wizard prompts for them, and document them in Environment Variables.

Send (email): to add a fourth sender, create a subfolder lib/sendProviders/<kind>/index.ts implementing SendProviderModule<'<kind>'>, add the literal to SendProviderKind in sendProviders/types.ts, and add one entry to the SEND_PROVIDERS registry in sendProviders/index.ts. There is no switch — the compile-time mapped-type check enforces the shape, so a missing method fails the build. No call sites change.

Design notes

• Caching: the LLM client is cached at module level (cachedClient in lib/llmProvider.ts) so each process resolves it once. There is no exported clear*Cache() — tests reset the module state with vi.resetModules(). The send provider has no cache at all: it uses the static SEND_PROVIDERS registry with the kind passed per call.
• Throwing on unknown: providerFor throws a descriptive Unknown send provider: <kind> error, so a typo fails loudly instead of silently falling back. (An unrecognized LLM_PROVIDER value — e.g. anthropic — silently falls through to the default OpenAI client with no base URL.)
• No secrets in getLLMConfig(): getLLMConfig() returns a snapshot of resolved settings with hasApiKey: boolean rather than the key itself, safe to log. (The send provider has no config getter — SendProviderModule exposes only kind, retryDelays, sendEmail, and categorizeError.)
• Embeddings: getEmbeddingModel() always resolves a single OpenAI-compatible embedding model (LLM_EMBEDDING_MODEL, default text-embedding-3-small) and throws only on a known dimension mismatch against EMBEDDING_DIMENSIONS; assertEmbeddingDimension enforces the same at write time. There is no separate embedding-provider env var.