Skip to content

Model catalog

For the administrator workflow, see Manage models. This reference specifies the version-1 JSON catalog schema and runtime rules.

Document format

A minimal catalog for a local OpenAI-compatible model server:

{
  "version": 1,
  "default_model": "local-assistant",
  "models": [
    {
      "id": "local-assistant",
      "label": "Local assistant",
      "provider": "local",
      "wire_api": "openai_chat",
      "api_model": "your-server-model-id"
    }
  ]
}

Replace api_model with the model identifier served by your endpoint. This minimal entry declares a text-only model without reasoning; set input_modalities and the reasoning_* fields to match your model's capabilities.

Top-level field Required value
version Integer 1
default_model id of an enabled entry in models
models List of 1 to 1,000 model entries

Unknown fields are rejected at both the document and model levels.

Model fields

Field Default Meaning
id Required Stable Numi identifier stored on chats (1 to 256 characters, no whitespace or control characters)
label Required Display name in the UI (1 to 160 characters)
provider Required Provider configuration key; see Providers
wire_api openai_chat Wire protocol: openai_chat, openai_responses, or anthropic_messages
api_model null Upstream model ID sent to the provider; falls back to id when null
hint "" Short description shown in the model selector (up to 1,000 characters)
enabled true Whether users can select the model when its provider is configured
input_modalities [] Non-text input types (image, video, audio); text is always implied
context_window null Token context window (at least 1,024); overrides the provider fallback for history budgeting and compaction
supports_prompt_cache false Enables explicit prompt-cache markers where needed (for example, root cache_control on OpenRouter for Anthropic-style models). Discovery sets this automatically when a model advertises cache-write pricing
input_price null Public API list price in USD per million input tokens. With output_price set, replies show an estimated cost (≈) for providers that report tokens but no cost; OpenRouter models keep their reported cost. Chat Completions requests then ask for stream_options.include_usage, so leave prices empty for a provider that rejects that field
cached_input_price null Price in USD per million cached input tokens; falls back to input_price
output_price null Price in USD per million output tokens, reasoning included
supports_reasoning false Marks the model as supporting reasoning traces or effort controls
reasoning_mode none How reasoning is requested: native, effort, enabled, or none
preserve_reasoning false Replays stored reasoning back to the model on subsequent turns where the wire protocol supports it
reasoning_mandatory false Indicates that reasoning cannot be turned off for this model
reasoning_efforts [] Supported effort levels for this route (for example ["low", "medium", "high"])
reasoning_default_effort null Default effort level; must belong to reasoning_efforts when that list is non-empty
reasoning_default_enabled null Provider's default reasoning state (true, false, or null)

Validation and uniqueness rules

  • id values must be unique across the catalog.
  • Routes must also be unique by (provider, wire_api, api_model or id).
  • OpenRouter variant suffixes such as :floor, :nitro, or :free count as distinct routes during validation, while merge imports and packaged-default checks ignore the suffix when matching existing models.
  • Non-reasoning entries (supports_reasoning: false) must use reasoning_mode: "none" and cannot set preserve_reasoning, reasoning_mandatory, or effort lists.
  • Reasoning entries (supports_reasoning: true) must choose a reasoning_mode other than "none". Models with reasoning_mandatory: true cannot set reasoning_default_enabled: false.
  • anthropic_messages requires provider to be anthropic or anthropic_compat.

See Configure Reasoning Effort and Preserved Reasoning for how reasoning settings behave during a turn.

Packaged catalog and model states

The packaged defaults live in src/numi_chat/llm/models.default.json.

State Meaning
Configured The entry exists in the active catalog
Enabled enabled is true on the entry
Available The entry's provider has the required API key / base URL configured
Selectable Both Enabled and Available

Disabled entries remain resolvable for existing chats so history still loads and displays their label.

Catalog precedence and persistence

If the file at NUMI_CHAT_MODEL_CONFIG_PATH (default state/models.json, or /app/state/models.json in Docker) exists, it replaces models.default.json completely.

Admin changes validate the new catalog and write state/models.json atomically before swapping the in-memory snapshot. Invalid JSON at state/models.json prevents server startup, while a failed reload at runtime keeps the last valid snapshot active. API keys and base URLs are never written to the catalog file.