Skip to content

Connect Providers and Endpoints

A model provider supplies API credentials and a base URL, while a model catalog entry defines the model ID, capabilities, and wire protocol (openai_chat, openai_responses, or anthropic_messages). Both must be in place before a model appears in the chat selector.

The packaged default catalog uses OpenRouter. Setting OPENROUTER_API_KEY in .env and restarting the server unlocks those default models immediately. To connect additional providers, follow the steps below.

Connect a provider, match a model and adapter, and verify a chat reply.

1. Configure provider credentials in .env

Set the credentials and base URL for each provider you want to use in .env (never store API keys inside models.json):

Catalog provider API key variable Base URL variable Default base URL
openrouter OPENROUTER_API_KEY OPENROUTER_BASE_URL https://openrouter.ai/api/v1
crowbot CROWBOT_API_KEY CROWBOT_BASE_URL https://api.crowbot.sh/v1
zai Z_AI_API_KEY Z_AI_BASE_URL https://api.z.ai/api/coding/paas/v4
openai OPENAI_API_KEY OPENAI_BASE_URL https://api.openai.com/v1
responses RESPONSES_API_KEY RESPONSES_BASE_URL (must be set)
anthropic ANTHROPIC_API_KEY ANTHROPIC_BASE_URL https://api.anthropic.com
anthropic_compat ANTHROPIC_COMPAT_API_KEY ANTHROPIC_COMPAT_BASE_URL (must be set)
inception INCEPTION_API_KEY INCEPTION_BASE_URL https://api.inceptionlabs.ai/v1
mistral MISTRAL_API_KEY MISTRAL_BASE_URL https://api.mistral.ai/v1
local LOCAL_AI_API_KEY (optional) LOCAL_AI_BASE_URL http://localhost:8080/v1

An administrator can also connect their personal ChatGPT Plus or Pro subscription without an API key or catalog entry; see Use your ChatGPT subscription.

For a local OpenAI-compatible server (such as llama-server, vLLM, Ollama, or LM Studio) listening on port 8080:

LOCAL_AI_BASE_URL="http://localhost:8080/v1"
LOCAL_AI_API_KEY="sk-no-key-required"

When running Numi inside Docker, localhost inside the container refers to the container itself; point LOCAL_AI_BASE_URL at http://host.docker.internal:8080/v1 or the host's LAN/bridge IP so the container can reach your inference server.

Restart Numi after editing .env (for Docker, run docker compose up -d so updated environment variables are applied).

2. Add or enable models in the catalog

  1. Sign in as an administrator and open Models (/models).
  2. Click Add Model, pick your provider, and either use Find on provider to discover available models or fill in the fields manually.
  3. Select the wire protocol accepted by that endpoint:
  4. openai_chat for OpenRouter, Crowbot, Z.AI, Inception, Mistral, and standard Chat Completions endpoints.
  5. openai_responses for OpenAI Responses (openai or responses).
  6. anthropic_messages for Anthropic Messages (anthropic or anthropic_compat).
  7. Set the upstream model identifier in api_model (if it differs from the catalog id), enable the entry, and click Save.
  8. Optionally set the model as the catalog default for new chats.

Provider discovery notes

For OpenRouter, Crowbot, and Mistral, provider discovery automatically populates reasoning capabilities, supported effort levels, image input support, and context windows. For Mistral reasoning models (which support thinking on at high or off at none), the composer automatically presents an Instant/Thinking toggle, and the adapter strips any non-Mistral message fields before sending requests.

See Manage models for details on editing, importing, and refreshing model metadata.

3. Check background model choices

Background memory extraction and automatic chat titles use their own model settings (MEMORY_SUMMARY_MODEL and CHAT_TITLE_MODEL), which default to deepseek/deepseek-v4.1-flash-1017.

If your catalog does not include that model or you want background tasks to follow your catalog's default_model, set both to inherit in .env:

MEMORY_SUMMARY_MODEL=inherit
CHAT_TITLE_MODEL=inherit

You can also pick dedicated overrides for both tasks at runtime on the admin Models page (saved UI overrides take precedence over .env). Background jobs require a catalog model configured with wire_api: "openai_chat".

4. Verify with a test message

Start a new chat, pick your newly added model from the selector, and send a short prompt to confirm the credentials, base URL, and upstream api_model work end to end.

Troubleshooting

Symptom What to check
Model is missing from the chat selector Confirm the catalog entry is enabled: true and its provider has credentials configured in .env.
Provider returns 401 / 403 Double-check the API key and base URL in .env and restart the service.
Connection refused for a local model Confirm the inference server is running and reachable from the Numi process or container.
Upstream rejects the model ID Verify that api_model matches the exact model identifier expected by the provider.
Provider rejects reasoning parameters Use Refresh provider metadata on the Models page or verify the entry's wire_api, reasoning_mode, and preserve_reasoning settings.
Chat works, but titles or memory fail Check CHAT_TITLE_MODEL / MEMORY_SUMMARY_MODEL and any saved overrides on the admin Models page.

See the Configuration reference for all provider settings.