Connect your AI
Pick the AI backend every Owlat AI feature uses — a hosted provider via an API key (OpenAI, Anthropic, Google, Azure OpenAI, OpenRouter) or a model you host yourself (Ollama, vLLM, llama.cpp) — plus the local-by-default embeddings that make retrieval work under any choice.
Every AI feature in Owlat — drafting replies, the assistant, summaries, translation, classification, the knowledge graph, semantic search — runs on one AI backend you choose once. Set it from Settings → AI Provider; there is no per-feature wiring. Swapping backends later is a single settings edit.
Owlat resolves AI on two independent planes:
- a language plane — all text generation, and
- an embedding plane — semantic search and the knowledge graph.
They are chosen separately, so a language provider that has no embeddings API (Anthropic, for example) still gets working retrieval from the local embedder.
If you set LLM_* environment variables when self-hosting, those remain the deployment fallback and keep working with zero UI. The moment you save a provider in Settings → AI Provider, that stored per-org config takes over. Saving a provider is admin-only (organization:manage) and is recorded in the audit log.
The language plane
Open Settings → AI Provider → Language model and pick a provider. You can point at a hosted provider with an API key, or at a model you host yourself.
Hosted providers (API key)
Choose the provider, paste an API key, and pick a capable model (hard tasks — reasoning, long drafts) and a fast model (quick tasks — classification, short replies).
| Provider | What it is | Model ids look like |
|---|---|---|
| OpenAI | Hosted GPT models | gpt-4o, gpt-4o-mini |
| Anthropic (Claude) | Hosted Claude models | claude-opus-4-8, claude-haiku-4-5 |
| Google (Gemini) | Hosted Gemini models | gemini-2.5-pro, gemini-2.5-flash |
| Azure OpenAI | GPT models from your Azure resource | your deployment names |
| OpenRouter | One key, many upstream models | anthropic/claude-sonnet-4-5, openai/gpt-4o |
For Azure OpenAI, the model id is the name you gave the deployment in the Azure portal — not a canonical id like gpt-4o. Paste your resource base URL (https://<resource>.openai.azure.com/openai) alongside the key; the base-URL field is shown by default for Azure because it is required.
For OpenRouter and local providers, save the config, then click Load available models to replace the free-text field with the exact catalog your provider serves.
Your API key is sent once over TLS, encrypted at rest (AES-256-GCM), and never shown again — the settings page only ever displays a masked preview and a "key is set" indicator. Use Test connection after saving to confirm the credentials work.
Local / self-hosted models (no key)
Pick Local / self-hosted (OpenAI-compatible) to run the language plane on your own hardware — Ollama, vLLM, or llama.cpp. There is no API key: just a base URL pointing at your server's OpenAI-compatible endpoint (Ollama's default is http://localhost:11434/v1).
# Example: serve a language model with Ollama
ollama pull llama3.1
ollama serve # exposes an OpenAI-compatible API on :11434
Then set the base URL to your server and, optionally, Load available models to see what it serves.
The embedding plane — local by default
Semantic search and the knowledge graph turn text into vectors with an embedding model. This plane is local by default: a bundled local embedding model ships with your deployment, so retrieval works out of the box under any language choice — including providers, like Anthropic, that offer no embeddings API. Most deployments never touch this section.
Vectors from different embedding models are not comparable. If you switch the embedding provider or model, Owlat bumps an internal version and prompts you to re-index — old and new vectors are never silently mixed. Prefer to leave embeddings on the local default unless you have a specific reason to change them.
If you do need a hosted embedder, expand Advanced: use a hosted embedder instead and choose OpenAI or Google (each with its own encrypted key), or a custom OpenAI-compatible embeddings server via a base URL.
Which model gets used?
Owlat routes each call to a tier automatically:
- Fast — classification, extraction, guardrails, summarization.
- Capable — drafting replies, planning multi-step actions.
You set the model id for each tier per provider. Spend is tracked per feature and per provider backend in Settings → System so you can see where your tokens go.
Turning AI on
The AI features themselves are gated behind the master ai flag and its sub-flags (ai.agent, ai.assistant, ai.knowledge, …). A saved provider config satisfies the "AI is configured" requirement, so once you connect a provider you can enable the AI features from Settings → Features. See the AI guides for what each feature does.