Knowledge Graph Architecture

Every organization accumulates knowledge through communication: customer preferences, internal decisions, project context, relationship history. The Knowledge Graph captures this knowledge as typed, searchable, decaying entries — not a write-only log, but a living system that stays accurate as the organization evolves.

Single-deployment isolation

Owlat runs one organization per deployment (see the runtime invariant in apps/api/convex/lib/sessionOrganization.ts). Because of that, knowledge isolation happens at the deployment boundary, not through per-query filtering: there is no organizationId field on knowledgeEntries or knowledgeRelations, and no query scopes by org. Every entry in a deployment belongs to that deployment's single org.

Storage model

The Knowledge Graph is built on Convex tables — not a separate graph database. The schema reserves a Convex vector index for semantic search, and indexed joins handle relationship traversal. This keeps the self-hosted stack simple: no Neo4j, no Pinecone, no additional services.

Knowledge entries

Every piece of organizational knowledge is a typed entry:

Each entry has:

• Content — the knowledge itself (title + detailed content)
• Source attribution — where this knowledge came from (email, chat, manual, file, agent_extracted)
• Entity links — optional connections to contacts (contactIds) and a conversation thread (threadId)
• Embedding — a 1536-dimension vector plus the embeddingModel and embeddingGeneratedAt that produced it, so stale embeddings can be re-generated
• Confidence score — how reliable this knowledge is (0–1), with a lastValidatedAt timestamp
• Expiration — optional expiresAt TTL for time-sensitive facts

Knowledge relations

Entries connect to each other through typed edges:

Relations are stored today (createRelation in graph.ts, indexed by_from / by_to), and getEntry returns an entry alongside its incoming and outgoing relations. Relation traversal also ships now: behind ai.knowledge.graphRetrieval, retrieval walks these edges to find supporting context and flag contradictions — see Graph-augmented retrieval below.

Extraction

The extractor (extractFromMessage in apps/api/convex/knowledge/extraction.ts) is a Node internalAction that does three things:

extractFromMessage(inboundMessageId)
  0. Idempotency guard: countBySource → early-return if already extracted
  1. LLM extraction: single structured-output call producing entries[]
  2. Embed each entry (text-embedding-3-small)
  3. saveEntry → insert into knowledgeEntries (dedup by source + title)

A companion extractFromFile mirrors this for processed documents (see the File System vision). A deterministic relates_to structural linker runs after store (scheduled by the extractor); saveEntry dedupes by source + title and by cross-source contentHash (matching contact scope). The inline before-store contradiction-check and supports / contradicts / supersedes inference remain part of the vision.

LLM extraction

Extraction uses one structured-output call against the extract model role, producing a flat entries[] array:

const extractionSchema = z.object({
  entries: z.array(z.object({
    type: z.enum(['fact', 'decision', 'event', 'preference', 'goal', 'relationship', 'action_item']),
    title: z.string(),
    content: z.string(),
    confidence: z.number().min(0).max(1),
    tags: z.array(z.string()).optional(),
  })),
})

const { object: extraction } = await runLlmObject({
  model: getLLMProvider('extract'),
  schema: extractionSchema,
  prompt: `Extract organizational knowledge from this email message...`,
  temperature: 0.1,
})

Each extracted entry is embedded and stored via internal.knowledge.graph.saveEntry with sourceType: 'agent_extracted' and sourceId set to the originating inboundMessageId (which both the live extractor and the backfill job use for idempotency, via countBySource and the by_source index).

Retrieval

The Knowledge Graph serves both the in-product browser and live agent retrieval today.

Full-text search (shipping)

Used by the in-product browser. The public search query runs Convex full-text search over searchableText, optionally filtered by entry type:

ctx.db
  .query('knowledgeEntries')
  .withSearchIndex('search_knowledge', (q) => {
    let sq = q.search('searchableText', args.searchQuery)
    if (args.entryType) sq = sq.eq('entryType', args.entryType)
    return sq
  })
  .take(limit)

The browse-by-type path (listByType) reads through the by_entry_type index. Both back the useKnowledgeGraph composable (apps/web/app/composables/useKnowledgeGraph.ts).

Contact-scoped retrieval (shipping)

Convex cannot index array fields directly, so knowledgeEntries.contactIds is mirrored into a dedicated junction table — knowledgeEntryContacts (apps/api/convex/schema/knowledge.ts), one row per (entryId, contactId) pair with a by_contact index. getByContact queries that index and hydrates the matching entries, so the lookup is complete and O(matches) — no truncation, no in-memory scan:

const links = await ctx.db
  .query('knowledgeEntryContacts')
  .withIndex('by_contact', (q) => q.eq('contactId', args.contactId))
  .collect()

const entryMap = await batchGet(ctx, links.map((link) => link.entryId))

return [...entryMap.values()]
  .filter((e) => e !== null && !(e.expiresAt !== undefined && e.expiresAt < now))
  .sort((a, b) => b.createdAt - a.createdAt)
  .slice(0, args.limit ?? 20)

Semantic search (shipping)

semanticSearch (apps/api/convex/knowledge/retrieval.ts) is now a real internalAction — vector search lives on the action context (ctx.vectorSearch), so the seam moved out of graph.ts into its own retrieval module. It accepts either a queryText (embedded here with text-embedding-3-small) or a pre-computed embedding, plus a required scopeToContact argument (a contactId, 'org-general-only', or 'org-wide') for contact-isolation post-filtering. Rather than a single vector pass it is hybrid: it runs two legs — a vector search over the vector_knowledge index and a full-text search (internal.knowledge.graph.ftsRankedIds) — and fuses their rankings with Reciprocal Rank Fusion (degrading to vector-only when there is no query text). It then hydrates the fused order to full documents via internal.knowledge.graph.getByIds, filters by contact scope, and annotates each survivor with its similarity _score:

// Leg 1 — semantic vector search
const hits = await ctx.vectorSearch('knowledgeEntries', 'vector_knowledge', {
  vector,
  limit: fetchLimit,
  filter: args.entryType ? (q) => q.eq('entryType', args.entryType) : undefined,
})
const vectorRanked = hits.map((h) => h._id)

// Leg 2 — full-text search (only when we have query text)
let ftsRanked = []
if (queryText) {
  ftsRanked = await ctx.runQuery(internal.knowledge.graph.ftsRankedIds, {
    queryText,
    entryType: args.entryType,
    limit: fetchLimit,
  })
}

// Fuse the two rankings (vector-only when FTS is empty), then hydrate
const fusedIds = reciprocalRankFusion([vectorRanked, ftsRanked])
const entries = await ctx.runQuery(internal.knowledge.graph.getByIds, {
  ids: fusedIds,
})

The vector_knowledge index (filterField ['entryType']) backs this. The old graph.ts semanticSearch internalQuery that returned recent rows is gone — getByIds replaced it as the document-hydration seam.

Graph-augmented retrieval: seed-then-expand (shipping)

The hybrid fusion above produces a flat nearest-neighbour set. When the caller passes expandGraph: true — set from the ai.knowledge.graphRetrieval feature flag by both the assistant searchKnowledge tool (apps/api/convex/assistant/tools.ts) and the agent context_retrieval step (apps/api/convex/agent/steps/context_retrieval/index.ts) — semanticSearch switches to seed-then-expand: the top contact-visible hits become seeds, and expandNeighbors (apps/api/convex/knowledge/graphTraversal.ts) walks one to two hops along knowledgeRelations edges to pull in the connected subgraph. lib/graphRank.ts then re-ranks the union — folding edge weight (RELATION_WEIGHTS), hop distance, and the original fused score — before hydrating in ranked order:

// Flat path OR graph-augmented — expandGraph is the kill switch.
let returned: ScoredKnowledgeEntry[];
if (args.expandGraph && visible.length > 0) {
  try {
    returned = await expandAndRank(ctx, { visible, vectorRanked, ftsRanked, scope, entryType, limit, hops: args.hops, neighborBudget: args.neighborBudget });
  } catch {
    returned = visible.slice(0, limit); // fail soft to flat
  }
} else {
  returned = visible.slice(0, limit);
}

Each expanded entry can carry three annotations (absent on the flat path):

• _via — the typed relationships that connected it to a seed within the returned subgraph, so a consumer can render why it surfaced.
• _stale — it is the target of a supersedes edge (a newer entry replaces it); the agent briefing prefixes it so the drafter treats it cautiously.
• _caveat — it is an endpoint of a contradicts edge; kept in the result but flagged as disputed rather than trusted outright.

The whole path fails soft: any traversal error falls back to the flat visible slice rather than dropping retrieval, and with the flag off the code takes the flat branch byte-for-byte.

The agent reads the graph

The live context_retrieval step (apps/api/convex/agent/steps/context_retrieval/index.ts) now folds semantic knowledge into its briefing. Alongside the contact profile, recent activities, thread history, and the current message, it builds a query from the inbound subject + body and runs two action-backed retrievals behind the step's token budget:

• internal.knowledge.retrieval.semanticSearch (limit knowledgeEntryLimit: 10) → a [KNOWLEDGE] section of the most relevant typed entries, each with its type and confidence
• internal.semanticFileProcessing.semanticSearch (limit fileLimit: 3) → a [RELEVANT FILES] section of relevant source documents (see the File System vision)

The knowledgeEntryLimit budget constant is now wired. The three compaction tiers (normal / compacted / emergency) still bound the assembled briefing. See the Agent Pipeline for how context retrieval fits the rest of the pipeline.

Planned: memory as tools

Extraction (above) is passive — it runs after a message is processed. The vision is for the agent to also actively save and recall knowledge during pipeline execution, by exposing tool definitions it can call.

Active save

When the agent discovers something important during a conversation — a new fact, an updated preference, a commitment — it would persist it immediately during the action-planning step (Step 3):

const saveKnowledge = tool({
  description: 'Save a piece of organizational knowledge discovered during this conversation',
  parameters: z.object({
    type: z.enum(['fact', 'decision', 'event', 'preference', 'goal', 'relationship', 'action_item']),
    title: z.string(),
    content: z.string(),
    contactId: z.string().optional(),
    confidence: z.number().min(0).max(1),
    expiresInDays: z.number().optional(),
  }),
  execute: async (args) => {
    // Would run dedup + contradiction check before storing
    return await ctx.runMutation(internal.knowledge.graph.saveEntry, { /* ... */ })
  },
})

Active recall

During draft generation (Step 4), the agent would explicitly query the graph for relevant context beyond what context retrieval already surfaced. The vector-search seam it would call (internal.knowledge.retrieval.semanticSearch) is real and shipping today — what's missing is exposing it as a tool the LLM can invoke on demand:

const recallKnowledge = tool({
  description: 'Search organizational knowledge for information relevant to the current task',
  parameters: z.object({
    query: z.string(),
    contactId: z.string().optional(),
    type: z.enum([/* entry types */]).optional(),
    limit: z.number().default(5),
  }),
  execute: async (args) => {
    // The underlying vector search already exists; this would wrap it as a tool
    return await ctx.runAction(internal.knowledge.retrieval.semanticSearch, { queryText: args.query, /* ... */ })
  },
})

Both build on the vector search above, which is already live — only the tool wrappers remain.

Decay and maintenance

The Knowledge Graph is not append-only. Stale knowledge degrades over time.

Confidence decay (shipping)

A scheduled Convex cron runs every 24 hours — crons.interval('knowledge graph maintenance', { hours: 24 }, internal.knowledge.maintenance.runDecay) (apps/api/convex/crons.ts). Each runDecay batch processes a batch of entries and does two things (apps/api/convex/knowledge/maintenance.ts):

1. Expiration — delete entries past their expiresAt timestamp, draining their relations first (relation deletion is capped per entry so a heavily-connected node doesn't blow the transaction budget; leftovers are picked up on the next run)
2. Time decay — reduce each entry's confidence based on its per-type decay rate and the days since lastValidatedAt, with a 0.1 floor. The decay factor is modulated by a usage-recency access boost (via accessCount / lastAccessedAt, bumped whenever an entry is recalled), so frequently-recalled entries decay more slowly

A separate daily dedup-merge cron (knowledge graph dedup → runKnowledgeDedup) runs alongside runDecay — see the Extraction section above.

Knowledge types decay at different rates

Schema

Defined in apps/api/convex/schema/knowledge.ts.

knowledgeEntries

knowledgeEntries: defineTable({
  entryType: v.union(
    v.literal('fact'),
    v.literal('decision'),
    v.literal('event'),
    v.literal('preference'),
    v.literal('goal'),
    v.literal('relationship'),
    v.literal('action_item')
  ),
  title: v.string(),
  content: v.string(),
  sourceType: v.union(
    v.literal('email'),
    v.literal('chat'),
    v.literal('manual'),
    v.literal('file'),
    v.literal('agent_extracted')
  ),
  sourceId: v.optional(v.string()),
  contactIds: v.optional(v.array(v.id('contacts'))),
  threadId: v.optional(v.id('conversationThreads')),
  embedding: v.array(v.float64()), // 1536 dims (text-embedding-3-small)
  embeddingModel: v.optional(v.string()),
  embeddingGeneratedAt: v.optional(v.number()),
  confidence: v.number(), // 0-1
  lastValidatedAt: v.number(),
  accessCount: v.optional(v.number()), // usage signal — bumped on recall
  lastAccessedAt: v.optional(v.number()), // usage signal — modulates decay
  expiresAt: v.optional(v.number()),
  tags: v.optional(v.array(v.string())),
  searchableText: v.optional(v.string()),
  contentHash: v.optional(v.string()), // sha256 fingerprint — cross-source dedup leg in saveEntry
  createdAt: v.number(),
  updatedAt: v.number(),
})
  .index('by_entry_type', ['entryType'])
  .index('by_created_at', ['createdAt'])
  .index('by_thread', ['threadId'])
  .index('by_source', ['sourceType', 'sourceId'])
  .index('by_content_hash', ['contentHash'])
  .searchIndex('search_knowledge', {
    searchField: 'searchableText',
    filterFields: ['entryType'],
  })
  .vectorIndex('vector_knowledge', {
    vectorField: 'embedding',
    dimensions: 1536,
    filterFields: ['entryType'],
  })

knowledgeRelations

knowledgeRelations: defineTable({
  fromEntryId: v.id('knowledgeEntries'),
  toEntryId: v.id('knowledgeEntries'),
  relationType: v.union(
    v.literal('supports'),
    v.literal('contradicts'),
    v.literal('supersedes'),
    v.literal('relates_to'),
    v.literal('causes'),
    v.literal('blocks')
  ),
  confidenceTag: v.union( // coarse, sortable companion to `confidence`
    v.literal('extracted'),
    v.literal('inferred'),
    v.literal('ambiguous')
  ),
  confidence: v.number(), // 0-1
  weight: v.optional(v.number()), // edge strength for graph-augmented retrieval
  provenance: v.union( // strongest first: manual > deterministic > llm
    v.literal('deterministic'),
    v.literal('llm'),
    v.literal('manual')
  ),
  rationale: v.optional(v.string()),
  createdAt: v.number(),
  updatedAt: v.number(),
})
  .index('by_from', ['fromEntryId'])
  .index('by_to', ['toEntryId'])
  .index('by_pair', ['fromEntryId', 'toEntryId'])
  .index('by_confidence_tag', ['confidenceTag'])