Audience Internals
Backend reference for contact resolution, the double opt-in lifecycle, topic subscription, the conditions registry, and segment evaluation.
This is the developer reference for the audience/contacts layer of the Convex backend: how a Contact is found-or-created, how double opt-in (DOI) is driven, how topic memberships are written, and how segments and automation conditions are evaluated. Every "given an identifier, find the Contact" and "does this Contact match" path in the codebase routes through the modules described here. For the product-facing view of these features, see Contacts, Topics, and Segments.
The modules live under apps/api/convex/contacts/, apps/api/convex/topics/, apps/api/convex/conditions/, and apps/api/convex/lib/. Several carry an ADR; the ADRs are the canonical design record and the source comments cite them by path (docs/adr/00xx-*.md).
Contact resolution & creation
apps/api/convex/contacts/resolution.ts is the single find-or-create primitive. Every intake path — inbound email, channel webhook, bulk import, HTTP API, automation trigger, form submission — reaches a Contact through it.
The lookup is uniform: every Contact is keyed by contactIdentities.by_identifier (the (channel, identifier) index), never by contacts.by_email. For the email channel, contacts.email is denormalized off the primary identity row so legacy reads of contact.email keep working, but the identity row is the lookup primitive. Email identifiers are lowercased; phone-derived channels (sms, whatsapp, phone) are kept verbatim — callers normalize to E.164 first.
The exported helper resolveContact(ctx, signal) (and its wire mutation resolve) forks on mode:
| Mode | On match | On no match |
|---|---|---|
strict | throw ALREADY_EXISTS | create |
upsert | return matched id, no field update | create |
merge | patch fields where the new value is non-empty (existing wins for empty/undefined) | create |
ResolveResult is { contactId, action } where action is 'matched' | 'created' | 'updated'. The resolution module is deliberately effect-free: it owns the identity-row write on create, the searchableText computation, and the soft-delete filter on lookup — but it does not log activity, fan out automation triggers, or maintain the contact count. Those stay with callers, branched on action.
Channels and sources
Two enumerations bound the inputs (defined in resolution.ts):
ChannelKind:email,sms,whatsapp,phone,generic,chat.ContactSource:api,import,form,transactional,inbound. This becomescontacts.sourceand is recorded in thecreatedactivity's metadata.
The creation effect bundle
Single-create callers do not call resolveContact directly — they go through createContact in apps/api/convex/contacts/creation.ts, which wraps the effect-free primitive and, only on action === 'created', fires the uniform created-effect bundle:
incrementContactCount(ctx, 1)— keepcachedContactCounttrue.- The
contact_createdautomation trigger. - A
createdContact activity row tagged withmetadata.source. - A
contact.createdcustomer webhook fanout viascheduleFanout(email channel only).
The bulk Contact import module is the sole exception: it calls resolveContact directly and batches one incrementContactCount per page. Keeping these effects in this layer above the still-effect-free primitive is what lets import's batched count and the single-create bundle coexist without double-counting.
On soft-delete, deleteIdentitiesForContact hard-deletes every contactIdentities row for the Contact immediately (not after the 30-day retention window). The identifier — the privacy-sensitive datum — becomes reclaimable on day one, so creating a fresh Contact for a previously-used address never collides.
Double opt-in lifecycle
apps/api/convex/contacts/doiLifecycle.ts is the single writer of contacts.doiStatus and its companion fields (doiConfirmationToken, doiTokenExpiresAt, doiConfirmedAt, doiAttestedSource). It implements a three-state machine with these legal edges:
| From | Legal transitions |
|---|---|
not_required | → pending |
pending | → confirmed |
confirmed | terminal (no edges) |
There are two entry points: transition (keyed by contactId) and transitionByConfirmationToken (keyed by the URL token, used by the customer-facing confirm endpoints). Both return a TransitionOutcome — duplicate, illegal, terminal, expired, and not-found cases are reported, never thrown:
reason | Meaning |
|---|---|
contact_not_found | No Contact for the id |
token_not_found | No Contact holds that confirmation token |
token_expired | Token past doiTokenExpiresAt |
illegal_edge | Transition not allowed from the current state |
terminal | Already in a state with no outgoing edges |
Transitions are reducer-driven: a reducer returns { patch, effects, applied }, and the runner is the only place that touches the DB and the scheduler. Effects are:
send_confirmation_email— schedules the confirmation email. Only fires when the caller supplies asiteUrland the Contact has anemail(admin imports that pre-confirm out-of-band leavesiteUrlabsent).fire_topic_subscribed_triggers— at confirm time, fans out to every DOI-required Topic membership the Contact currently holds.contact_activity— onetopic_confirmedrow per DOI-required membership, plus adoi_attestedrow on the admin-attest path.audit_log— fires only on the admin-attest path (doi.admin_attested).
Tokens have a 7-day TTL (DOI_TOKEN_TTL_MS). A separate operation, refreshPendingToken, regenerates the token and re-sends the email without changing doiStatus — it refuses with not_pending if the Contact is not currently pending. It lives in this module so every write to the DOI fields goes through one file.
Admin-attest
When a Contact was already DOI-confirmed at a source platform (Mailchimp, Klaviyo, Stripe, a trusted CSV), the import path can attest that out-of-band. The admin-attest variant ({ to: 'confirmed', source: 'admin_attest', attestSource }) relaxes the otherwise-refused not_required → confirmed edge, records doiAttestedSource, and emits the audit_log + doi_attested activity. The token-keyed confirm path can never reach confirmed from not_required.
Topic subscription
apps/api/convex/topics/subscription.ts is the single writer of contactTopics. It also owns every maintenance of topics.cachedMemberCount, the DOI gate at subscribe time, and the per-source effect bundle on unsubscribe. There are five entry points keyed by shape:
| Entry point | Shape |
|---|---|
subscribe / subscribeMany | one topic, one-or-many contacts |
unsubscribe / unsubscribeMany | one topic, one-or-many contacts |
unsubscribeAllForContact | one contact, one-or-all topics |
SubscribeOutcome resolves to one of subscribed, pending_doi, or already_member. The pending_doi outcome carries the freshly-written doiToken so a caller recording a sibling row (the form module's formSubmissions.confirmationToken) avoids re-reading the Contact.
The DOI gate at subscribe time
When subscribing, DOI is required iff topic.requireDoubleOptIn === true and the caller did not pass skipDoi. If DOI is not in the way (or the Contact is already confirmed), the topic_subscribed trigger fires immediately. Otherwise the module hands off to the DOI lifecycle's transition(... to: 'pending') and returns pending_doi — it does not fire the trigger itself, because the lifecycle's fire_topic_subscribed_triggers effect at confirm time covers every DOI-required membership at once and would otherwise double-fire.
Source-conditional unsubscribe effects
Per-call unsubscribe effects are gated by the unsubscribe source. This table is the single place where "which side effects fire for which trigger" lives (effectFlagsForUnsubscribeSource):
| Source | Clear form confirmations | Increment campaign unsub stats | Fire topic.unsubscribed webhook |
|---|---|---|---|
public_email_link | yes | yes | yes |
preferences_page | yes | no | yes |
admin | no | no | no |
public_api | no | no | no |
Per-call effects (the cachedMemberCount patch, contact.updatedAt, form-clear, campaign-stats, webhook) fire once per call regardless of how many memberships are touched; per-membership effects (the membership row delete and the topic_unsubscribed activity row) fire N times. unsubscribeAllForContact is the entry point used by the public unsubscribe link and groups deletions per topic so each topic's cachedMemberCount is patched once.
Conditions registry
apps/api/convex/conditions/ holds the typed, pluggable registry that powers both Segments and the automation condition step. There are three condition kinds, each a ConditionTypeModule with parseCondition / preloadLookup / preloadLookupForContacts / evaluate (conditions/types.ts). preloadLookup is the whole-population preload; preloadLookupForContacts is the bounded per-contact variant used by the single-contact automation path and the per-page segment builder:
| Kind | Fields | Operators |
|---|---|---|
contact_property | any built-in (email, firstName, lastName, source) or custom property key | equals, not_equals, contains, not_contains, gt, lt, gte, lte, is_empty, not_empty, is_true, is_false |
email_activity | opened, clicked | is_true, is_false |
topic_membership | a topicId | equals, not_equals |
The registry's value is the batched evaluation seam (conditions/index.ts): rather than re-querying per Contact, preloadConditionsLookup groups the conditions by kind, hands each batch to its module, and stores a typed lookup keyed by kind. (preloadConditionsLookupForContacts is the bounded sibling that drives each module's preloadLookupForContacts for the per-contact paths.) Then evaluateOne(condition, contact, lookup) is a pure O(1) check. The preloads:
contact_propertyresolves custom-property IDs by key, then preloadscontactPropertyValuesfor those properties; built-in fields read straight off the Contact doc. String operators are case-insensitive; numeric operators coerce viaNumber().email_activityis evaluated directly off the denormalizedcontact.hasOpened/contact.hasClickedflags (maintained bycontactActivities/writer.ts); it has no preloaded lookup and performs no scan — an O(1) read off the already-loaded Contact row.topic_membershipreadscontactTopics.by_topicfor each referenced topic into a membershipSet.
parseCondition throws on a shape/operator/field violation — callers treat a parse failure as corrupt stored data, not user input, because conditions are storage-validated at write time.
Segment evaluation & the Listing engine
The layer above per-condition evaluation — filter normalization, the empty/AND/OR combine, and the live-Contact scan — lives in apps/api/convex/conditions/segmentMatch.ts. It is the single owner of "match a Contact population against a stored filter set", consolidating logic that previously drifted across five open-coded copies. apps/api/convex/conditions/index.ts re-exports a backward-compatible surface (evaluateCondition, evaluateSegmentCount, countLiveMatchesForSegments) from segmentMatch.ts that apps/api/convex/segments.ts imports via ./conditions.
The module has two layers:
- Pure core —
parseSegmentFilters+makeSegmentPredicate. These throw on corrupt filters. Empty conditions match every Contact; otherwise conditions combine with short-circuit AND/OR. This is the test surface and the seam the send path uses (it builds the predicate directly so it can interleave eligibility filtering in one walk). - Lenient async conveniences —
countLiveMatches,matchLiveContacts,countLiveMatchesForSegments. These bake in the soft-delete-excluding paginatedby_deleted_atscan and treat corrupt filters as a zero match — the posture the preview, count, and cron paths want.
A cron (crons.ts, every 30 minutes) calls refreshAllSegmentCounts to keep segments.cachedCount / cachedCountUpdatedAt fresh for the segments list UI. countLiveMatchesForSegments evaluates many segments in one pass: it flattens every segment's conditions into a single preloaded lookup and reuses one Contact scan.
The paged, sortable list of Contacts in the dashboard rides indexed reads via the resource Listing engine (see ADR-010: Listing Engine and the by_deleted_at_and_created_at index, where deletedAt leads so soft-deleted rows are dropped inside the index). Segment matching is a different operation: arbitrary filter predicates that cannot ride a single index, so it is a deliberately bounded full-table scan of live Contacts (see "Bounded scans" below).
Surprising behaviors
A few behaviors trip up new readers — they are intentional, but worth calling out.
DOI defaults to required
New Topics default to requireDoubleOptIn: true (topics/topics.ts, args.requireDoubleOptIn ?? true). A Contact subscribing to a fresh Topic therefore lands in pending and must confirm, unless the operator explicitly sets the Topic to single opt-in or the caller passes skipDoi. Newly-created Contacts themselves start at doiStatus: 'not_required' — DOI is gated at the Topic level, applied when a membership is created, not at Contact creation.
A form's doubleOptIn field is the union of form and topic controls: when form.doubleOptIn === true, the Forms submission path (forms/submission.ts) passes forceDoi: true into topics.subscription.subscribe, and the subscribe gate (topics/subscription.ts) requires DOI when topic.requireDoubleOptIn === true || forceDoi === true (and skipDoi !== true). So a form can require confirmation even on a single-opt-in topic, but it can never weaken a DOI-required topic. Segment audiences are never DOI-gated at all.
Emailless contacts
contacts.email is optional. A Contact that arrived via sms, whatsapp, phone, or generic has no email at all. Identity lookups go through contactIdentities.by_identifier, so an emailless Contact is fully addressable by its channel identifier. Code that needs an email defends accordingly: the topic.unsubscribed webhook falls back to '' when a Contact has no email (the payload contract requires a string), and ensureEmailIdentity is a no-op for emailless Contacts.
Bounded scans
Segment matching is a bounded full-table scan, not an indexed read — an arbitrary filter predicate cannot ride a single index. segmentMatch.ts scans live Contacts in bounded 500-row pages via the by_deleted_at index pinned to deletedAt === undefined (forEachLiveContact), so soft-deleted rows are dropped inside the index. (email_activity is no longer a scan — it is an O(1) denormalized-flag read off contact.hasOpened / contact.hasClicked.) Keep this in mind for very large audiences — the count cron amortizes the segment scan, and the campaign send path runs the same predicate once per send rather than per preview.