Message Flow and Persistence¶
Numi Chat maintains three distinct representations of a conversation: rows
persisted in SQLite (Message), the filtered message list prepared for an LLM
request (Agent.messages), and real-time JSON events streamed over
/ws/{chat_id}.
Message Lifecycle¶
- WebSocket submission: The browser sends a user message (with a
client-generated
request_idand any attachment filenames) over/ws/{chat_id}.web/routes/ws.pyverifies authentication, chat ownership, payload limits, and turn concurrency. - Optional tool activation gate: Before persisting the message,
services/chats/stream.pyasksagent/tool_group_router.pywhether any disabled optional tool groups strongly match the user's prompt. If so, it broadcasts atool_gateevent and waits briefly (DECISIONS_TOOL_GATE_TIMEOUT_SECONDS) for the user to accept or decline before continuing. Stopping while the card is open closes it without recording a decline and cancels the turn before the message is saved. - Agent initialization & user message save: The producer creates an
Agent, loads the chat's active model, settings, recent history, and latest compaction checkpoint, and saves the user message to SQLite. Anacceptedevent returns the saved message andrequest_idto the browser. - Streaming model step: The selected wire adapter streams
reasoning,content, andtool_preparingevents to the browser in real time. - Assistant step persistence: When the model step finishes, the agent saves
the assistant message (including any
tool_calls,reasoning_content,reasoning_details_json, andusage_json) before executing tools. - Tool execution:
ToolExecutorServicestreamstool_startandtool_endevents as tools finish, then saves thetoolresult messages in their original call order and loops back for the next model step. - Turn completion: Once the model produces a final reply with no further
tool calls, the agent emits
done(withmessage_id,citation_sources, and aggregatedusage), triggers background title generation or idle memory scheduling, and closes the turn withturn_end.
Message Structure (Message table)¶
| Column | Purpose |
|---|---|
role, content |
Message role (system, user, assistant, tool) and primary text |
content_parts_json |
Multimodal attachment references, the file list and PDF text first announced with the message (context_note), append-only model context (memory_note), recorded <current_time> (time_note), and request_meta (request_id, mode) |
reasoning_content |
Plain-text reasoning trace or summary for UI display and text-based replay |
reasoning_details_json |
Structured provider reasoning blocks (Anthropic signed thinking, OpenRouter details, OpenAI encrypted reasoning) |
tool_calls_json |
Assistant function calls (id, name, arguments) |
tool_call_id |
Pairs a role="tool" message with its assistant tool_call |
citation_sources_json |
Deduplicated [web:N] source metadata for citation chips |
usage_json |
Token counts and cost for an assistant step or metered tool call |
chat_id, created_at, deleted_at |
Chat foreign key, timestamp, and soft-deletion marker |
See Database Schema Reference for the complete table definition.
Preparing Messages for the Model¶
Before each LLM call, the agent transforms the loaded history into a provider-ready payload:
- Opening session context (
_first_user_context): Injects<active_model>,<optional_tools>, and<available_skills>at the first user message without mutating stored database rows. - Per-turn attachments and timestamps: Reconstructs permitted attachment
text/images and replays each user message's stored
context_note(new uploads and PDF text), append-onlymemory_noteupdates andtime_note(<current_time>... UTC</current_time>) so the prompt prefix stays byte-identical across turns. The context and memory notes are model-only: they are not part of message payloads and never enter a share snapshot. Only the owner's chat info panel summarizes what the latest memory note holds. - Reasoning & internal field filter (
prepare_messages): Strips internal app keys (_db_id,additional_data) and filtersreasoning_contentandreasoning_detailsbased on the target model'sprovider,supports_reasoning, andpreserve_reasoningsettings (see Preserved Reasoning). - Context budget & compaction: Summarizes older turns if auto-compaction is enabled and the quad-token estimate exceeds the model's threshold, or drops older complete turns if needed to fit within the history budget.
- Wire adapter conversion: Converts the prepared messages and active tool schemas into Chat Completions, Responses, or Anthropic Messages format.
Chat History Loading & Compaction Checkpoints¶
ChatContextService.load_history() in src/numi_chat/chat/context.py loads the
system message, the latest valid ChatCompaction summary (if any), and
non-deleted messages from the active response branch (ChatVersion):
- History loading is bounded by
PROMPT_MAX_HISTORY_MESSAGES(default1,000) and a character budget ofmax(PROMPT_MAX_HISTORY_CHARS, compact_threshold * 3)so models with large context windows can use their full capacity. - Complete
user -> assistant -> toolgroups are kept together so a request never starts with an orphaned tool result. - When older turns are omitted by the budget, the runtime inserts a brief notice so the model knows earlier messages were truncated.
WebSocket Streaming & Reconnect Replay¶
ConnectionManager (src/numi_chat/web/connection_manager.py) broadcasts JSON
events with monotonic event_id and turn_id identifiers:
Event type |
Meaning |
|---|---|
turn_start |
Turn producer started |
tool_gate |
Pre-turn router is offering optional tool groups for activation |
tool_gate_closed |
Pre-turn tool activation prompt was answered, timed out, or cancelled |
accepted |
User message was saved to SQLite (request_id, message) |
reasoning, content |
Incremental reasoning or answer text delta |
tool_preparing |
Tool call arguments are streaming from the model |
tool_start, tool_end |
Tool execution started / finished with output |
research_agent |
Live status update from a Deep Research sub-agent |
done |
Assistant step or final reply saved (message_id, citation_sources, usage) |
error |
Turn stopped due to an error, cancellation, or budget limit |
turn_end |
Turn producer finished |
queued_messages |
Updated list of pending queued inputs (chatqueuedmessage) |
title_update |
Generated or updated sidebar chat title |
See API Reference for full event schemas.
If a browser tab refreshes or reconnects while a turn is running, it sends its
last seen turn_id and event_id, and ConnectionManager replays the buffered
events (coalescing text deltas and keeping the latest state for snapshot_key
events like research_agent and tool_gate).
The snapshot limits count coalesced text rather than charging a separate
event envelope for every streamed chunk, so many small chunks do not
prematurely cancel an otherwise bounded reply.
Queued Input (chatqueuedmessage)¶
Messages submitted while a turn is already running are stored in
chatqueuedmessage outside the main transcript:
next_stepcontinuation: Accepted only at a safe boundary after all tool results in the current batch (and any active deep research wave) have been persisted. The agent emits an intermediatedoneevent, persists the queued user message (acceptedwithcontinuation: true), and continues the turn loop without resetting turn budgets.- Post-turn drain: Any remaining queued messages are drained in order after the final answer completes.
- Crash & error safety: Queued inputs stay in
chatqueuedmessageuntil accepted and saved as aMessagerow. If a turn is cancelled or errors out, unaccepted queued messages remain above the composer so the user can click Send now or edit/delete them.
Usage Accounting¶
Both LLM inference calls and metered web search/fetch calls record usage on
their respective Message rows (usage_json):
- OpenRouter reports the real cost of each request. For other providers the
cost is an estimate from the model's list prices (
input_price,cached_input_price,output_pricein the model catalog) and the token counts the provider reports; a model without prices records no cost. ChatGPT plan models are estimated the same way from public API prices. Estimates are flaggedestimatedandlist_priceand shown with≈. - Tool calls store their web API cost on the
role="tool"message'susage_jsonbefore any output truncation or citation deduplication. - In Deep Research mode,
research_pollattaches the sub-agents' accumulated token and tool spend onto the poll tool message'susage_json(and any unbilled spend at turn exit is recorded on a closing assistant record). - The
doneevent and saved chat overview sum assistant and tool usage across the turn, breaking outinferenceandtoolssubtotals while stripping all usage metadata from model-history replay, memory extraction, and public share snapshots.