Model catalog¶
For the administrator workflow, see Manage models. This reference specifies the version-1 JSON catalog schema and runtime rules.
Document format¶
A minimal catalog for a local OpenAI-compatible model server:
{
"version": 1,
"default_model": "local-assistant",
"models": [
{
"id": "local-assistant",
"label": "Local assistant",
"provider": "local",
"wire_api": "openai_chat",
"api_model": "your-server-model-id"
}
]
}
Replace api_model with the model identifier served by your endpoint. This
minimal entry declares a text-only model without reasoning; set
input_modalities
and the reasoning_* fields to match your model's capabilities.
| Top-level field | Required value |
|---|---|
version |
Integer 1 |
default_model |
id of an enabled entry in models |
models |
List of 1 to 1,000 model entries |
Unknown fields are rejected at both the document and model levels.
Model fields¶
| Field | Default | Meaning |
|---|---|---|
id |
Required | Stable Numi identifier stored on chats (1 to 256 characters, no whitespace or control characters) |
label |
Required | Display name in the UI (1 to 160 characters) |
provider |
Required | Provider configuration key; see Providers |
wire_api |
openai_chat |
Wire protocol: openai_chat, openai_responses, or anthropic_messages |
api_model |
null |
Upstream model ID sent to the provider; falls back to id when null |
hint |
"" |
Short description shown in the model selector (up to 1,000 characters) |
enabled |
true |
Whether users can select the model when its provider is configured |
input_modalities |
[] |
Non-text input types (image, video, audio); text is always implied |
context_window |
null |
Token context window (at least 1,024); overrides the provider fallback for history budgeting and compaction |
supports_prompt_cache |
false |
Enables explicit prompt-cache markers where needed (for example, root cache_control on OpenRouter for Anthropic-style models). Discovery sets this automatically when a model advertises cache-write pricing |
input_price |
null |
Public API list price in USD per million input tokens. With output_price set, replies show an estimated cost (≈) for providers that report tokens but no cost; OpenRouter models keep their reported cost. Chat Completions requests then ask for stream_options.include_usage, so leave prices empty for a provider that rejects that field |
cached_input_price |
null |
Price in USD per million cached input tokens; falls back to input_price |
output_price |
null |
Price in USD per million output tokens, reasoning included |
supports_reasoning |
false |
Marks the model as supporting reasoning traces or effort controls |
reasoning_mode |
none |
How reasoning is requested: native, effort, enabled, or none |
preserve_reasoning |
false |
Replays stored reasoning back to the model on subsequent turns where the wire protocol supports it |
reasoning_mandatory |
false |
Indicates that reasoning cannot be turned off for this model |
reasoning_efforts |
[] |
Supported effort levels for this route (for example ["low", "medium", "high"]) |
reasoning_default_effort |
null |
Default effort level; must belong to reasoning_efforts when that list is non-empty |
reasoning_default_enabled |
null |
Provider's default reasoning state (true, false, or null) |
Validation and uniqueness rules¶
idvalues must be unique across the catalog.- Routes must also be unique by
(provider, wire_api, api_model or id). - OpenRouter variant suffixes such as
:floor,:nitro, or:freecount as distinct routes during validation, whilemergeimports and packaged-default checks ignore the suffix when matching existing models. - Non-reasoning entries (
supports_reasoning: false) must usereasoning_mode: "none"and cannot setpreserve_reasoning,reasoning_mandatory, or effort lists. - Reasoning entries (
supports_reasoning: true) must choose areasoning_modeother than"none". Models withreasoning_mandatory: truecannot setreasoning_default_enabled: false. anthropic_messagesrequiresproviderto beanthropicoranthropic_compat.
See Configure Reasoning Effort and Preserved Reasoning for how reasoning settings behave during a turn.
Packaged catalog and model states¶
The packaged defaults live in src/numi_chat/llm/models.default.json.
| State | Meaning |
|---|---|
| Configured | The entry exists in the active catalog |
| Enabled | enabled is true on the entry |
| Available | The entry's provider has the required API key / base URL configured |
| Selectable | Both Enabled and Available |
Disabled entries remain resolvable for existing chats so history still loads and displays their label.
Catalog precedence and persistence¶
If the file at NUMI_CHAT_MODEL_CONFIG_PATH (default state/models.json, or
/app/state/models.json in Docker) exists, it replaces models.default.json
completely.
Admin changes validate the new catalog and write state/models.json atomically
before swapping the in-memory snapshot. Invalid JSON at state/models.json
prevents server startup, while a failed reload at runtime keeps the last valid
snapshot active. API keys and base URLs are never written to the catalog file.