Skip to content

Preserved Reasoning System

Numi Chat captures reasoning traces returned by a model, stores them in SQLite alongside the assistant message, and replays them on subsequent turns when the active model and wire protocol support it.

To adjust how much reasoning budget a model uses per chat, see Configure Reasoning Effort.

Display a trace, store it, and replay it if the model and API allow.

Three Stages of Reasoning Handling

Stage What Numi Chat does
1. Display Streams readable reasoning text or summaries (reasoning events) into a collapsible block in the browser.
2. Storage Persists plain-text reasoning_content and structured provider blocks (reasoning_details_json, such as cryptographic signatures or encrypted items) on the Message row in chats.db.
3. Replay Filters and formats stored reasoning fields via prepare_messages() before each request according to the target model's provider, wire_api, and preserve_reasoning settings.

Because prepare_messages() operates on a copy of the loaded history at request time, changing preserve_reasoning or switching models never mutates or erases the reasoning traces saved in SQLite.

Reasoning Modes

The catalog's reasoning_mode controls how reasoning is requested on the current turn, while preserve_reasoning controls whether returned reasoning is replayed on later turns:

reasoning_mode How reasoning is requested
effort Sends a named effort level (minimal, low, medium, high, xhigh, max), normalized to the model's reasoning_efforts list.
enabled Sends a boolean on/off reasoning toggle.
native The model reasons natively without an effort parameter (effort is fixed in the UI).
none The model does not support reasoning; reasoning parameters are omitted and historical reasoning fields are stripped.

Many models in the default catalog combine reasoning_mode: "effort" with preserve_reasoning: true. See Model Catalog Reference for per-model settings.

The Message Preparation Filter (llm/messages.py)

Before the wire adapter builds the outbound payload, prepare_messages() in src/numi_chat/llm/messages.py inspects each historical assistant message:

Route / Capability reasoning_content (plain text) reasoning_details (structured blocks)
supports_reasoning: false Stripped Stripped
Direct openai or inception provider Stripped Stripped
anthropic provider Stripped (native API requires signed blocks) Kept (thinking with signature and redacted_thinking)
anthropic_compat provider Kept as fallback when unsigned Kept (thinking and redacted_thinking)
openrouter with preserve_reasoning: true Kept Kept
openrouter with preserve_reasoning: false Stripped Kept on tool_calls messages for tool continuation; stripped otherwise
mistral with preserve_reasoning: true Kept (converted to a leading thinking content chunk) Stripped
chatgpt (admin ChatGPT subscription) Stripped (summary is display-only) Kept (openai.encrypted_reasoning items for matching api_model)
Other routes (zai, crowbot, local) Kept when preserve_reasoning: true Stripped

Wire Adapter Conversions (llm/adapters/)

After prepare_messages() filters the history, the active wire_api adapter formats the request for the upstream endpoint:

  • OpenAI Chat Completions (openai_chat.py):
  • Z.AI (provider: "zai"): Sends thinking: {"type": "enabled", "clear_thinking": false} and replays reasoning_content on assistant messages.
  • OpenRouter (provider: "openrouter"): Sends reasoning: {"effort": ...} or {"enabled": ...} and replays reasoning_content and reasoning_details when preserved.
  • Mistral (provider: "mistral"): Streams and parses thinking content chunks and replays preserved reasoning_content as a leading {"type": "thinking", "thinking": [{"type": "text", "text": ...}]} block on assistant messages.
  • Local endpoints (provider: "local"): Extracts <think>...</think> blocks from streamed content into reasoning_content for collapsible UI display.
  • OpenAI Responses (openai_responses.py):
  • Converts messages and tool calls into Responses input items and streams reasoning summaries.
  • For an admin's ChatGPT subscription (provider: "chatgpt"), requests include: ["reasoning.encrypted_content"], tags each returned openai.encrypted_reasoning item with the api_model that produced it, and replays those items (without server IDs, since store: false) ahead of that turn's text and function calls whenever the same model is used.
  • Anthropic Messages (anthropic_messages.py):
  • Stores each completed thinking block (with its cryptographic signature) and redacted_thinking block in reasoning_details, and replays them verbatim at the start of the assistant message's content array ahead of text and tool_use blocks.

Branching to Another Model

Because cryptographic signatures (anthropic) and encrypted reasoning items (chatgpt) only validate on the model that generated them, and replaying one model's plain-text reasoning to a different model can confuse its persona or reasoning style, Continue with another model copies the visible conversation, tool calls, tool results, and files into a new linked chat while leaving the original model's reasoning traces behind in the source chat.

Context Window and Compaction Trade-offs

Replaying reasoning traces across turns keeps the model's working state intact across tool loops and follow-up questions, at the cost of additional input tokens (counted in the request's quad-token estimate). When a conversation grows past the compaction threshold or history budget, older turns and their reasoning traces are summarized or dropped together. See Configure Auto-Compaction and Message Flow and Persistence.