Connect Providers and Endpoints¶
A model provider supplies API credentials and a base URL, while a model catalog
entry defines the model ID, capabilities, and wire protocol (openai_chat,
openai_responses, or anthropic_messages). Both must be in place before a
model appears in the chat selector.
The packaged default catalog uses OpenRouter. Setting OPENROUTER_API_KEY in
.env and restarting the server unlocks those default models immediately. To
connect additional providers, follow the steps below.
1. Configure provider credentials in .env¶
Set the credentials and base URL for each provider you want to use in .env
(never store API keys inside models.json):
Catalog provider |
API key variable | Base URL variable | Default base URL |
|---|---|---|---|
openrouter |
OPENROUTER_API_KEY |
OPENROUTER_BASE_URL |
https://openrouter.ai/api/v1 |
crowbot |
CROWBOT_API_KEY |
CROWBOT_BASE_URL |
https://api.crowbot.sh/v1 |
zai |
Z_AI_API_KEY |
Z_AI_BASE_URL |
https://api.z.ai/api/coding/paas/v4 |
openai |
OPENAI_API_KEY |
OPENAI_BASE_URL |
https://api.openai.com/v1 |
responses |
RESPONSES_API_KEY |
RESPONSES_BASE_URL |
(must be set) |
anthropic |
ANTHROPIC_API_KEY |
ANTHROPIC_BASE_URL |
https://api.anthropic.com |
anthropic_compat |
ANTHROPIC_COMPAT_API_KEY |
ANTHROPIC_COMPAT_BASE_URL |
(must be set) |
inception |
INCEPTION_API_KEY |
INCEPTION_BASE_URL |
https://api.inceptionlabs.ai/v1 |
mistral |
MISTRAL_API_KEY |
MISTRAL_BASE_URL |
https://api.mistral.ai/v1 |
local |
LOCAL_AI_API_KEY (optional) |
LOCAL_AI_BASE_URL |
http://localhost:8080/v1 |
An administrator can also connect their personal ChatGPT Plus or Pro subscription without an API key or catalog entry; see Use your ChatGPT subscription.
For a local OpenAI-compatible server (such as llama-server, vLLM, Ollama, or
LM Studio) listening on port 8080:
When running Numi inside Docker, localhost inside the container refers to the
container itself; point LOCAL_AI_BASE_URL at
http://host.docker.internal:8080/v1 or the host's LAN/bridge IP so the
container can reach your inference server.
Restart Numi after editing .env (for Docker, run docker compose up -d so
updated environment variables are applied).
2. Add or enable models in the catalog¶
- Sign in as an administrator and open Models (
/models). - Click Add Model, pick your provider, and either use Find on provider to discover available models or fill in the fields manually.
- Select the wire protocol accepted by that endpoint:
openai_chatfor OpenRouter, Crowbot, Z.AI, Inception, Mistral, and standard Chat Completions endpoints.openai_responsesfor OpenAI Responses (openaiorresponses).anthropic_messagesfor Anthropic Messages (anthropicoranthropic_compat).- Set the upstream model identifier in
api_model(if it differs from the catalogid), enable the entry, and click Save. - Optionally set the model as the catalog default for new chats.
Provider discovery notes¶
For OpenRouter, Crowbot, and Mistral, provider discovery automatically populates
reasoning capabilities, supported effort levels, image input support, and
context windows. For Mistral reasoning models (which support thinking on at
high or off at none), the composer automatically presents an
Instant/Thinking toggle, and the adapter strips any non-Mistral message fields
before sending requests.
See Manage models for details on editing, importing, and refreshing model metadata.
3. Check background model choices¶
Background memory extraction and automatic chat titles use their own model
settings (MEMORY_SUMMARY_MODEL and CHAT_TITLE_MODEL), which default to
deepseek/deepseek-v4.1-flash-1017.
If your catalog does not include that model or you want background tasks to
follow your catalog's default_model, set both to inherit in .env:
You can also pick dedicated overrides for both tasks at runtime on the admin
Models page (saved UI overrides take precedence over .env). Background
jobs require a catalog model configured with wire_api: "openai_chat".
4. Verify with a test message¶
Start a new chat, pick your newly added model from the selector, and send a
short prompt to confirm the credentials, base URL, and upstream api_model work
end to end.
Troubleshooting¶
| Symptom | What to check |
|---|---|
| Model is missing from the chat selector | Confirm the catalog entry is enabled: true and its provider has credentials configured in .env. |
Provider returns 401 / 403 |
Double-check the API key and base URL in .env and restart the service. |
| Connection refused for a local model | Confirm the inference server is running and reachable from the Numi process or container. |
| Upstream rejects the model ID | Verify that api_model matches the exact model identifier expected by the provider. |
| Provider rejects reasoning parameters | Use Refresh provider metadata on the Models page or verify the entry's wire_api, reasoning_mode, and preserve_reasoning settings. |
| Chat works, but titles or memory fail | Check CHAT_TITLE_MODEL / MEMORY_SUMMARY_MODEL and any saved overrides on the admin Models page. |
See the Configuration reference for all provider settings.