Message Flow and Persistence

Numi Chat keeps three related representations: messages stored in the database, the history prepared for a model request, and events streamed to the browser. They serve different purposes. A token visible in the browser is not necessarily committed yet, and a stored message is not necessarily included in the next model request.

Message Lifecycle

1. The browser sends a message and attachment references over /ws/{chat_id}. The route checks authentication, chat ownership, payload limits, and whether a turn can start.
2. A producer thread creates an Agent and loads the chat's model, settings, recent history, and any compaction checkpoint.
3. The agent saves the user message before requesting a model response. When the client supplied a request ID, accepted identifies the saved message.
4. The adapter streams answer, reasoning, and tool-call events. The browser can render these before the assistant message is complete.
5. At the end of a model step, the agent saves the assembled assistant message. A response with tool calls is saved before those tools run.
6. Tool results stream as they finish, then are saved in call order. The next model step receives the assistant tool calls and matching results.
7. A final answer without more tool calls produces done. The producer emits turn_end when the turn exits, including after errors.

The request ID also lets the server recognize a user message already saved when a client retries. Streaming and persistence are coordinated, but a process crash can still lose output that has not reached a save boundary.

Message Structure

Field	Purpose
role, content	Message author and main text
content_parts_json	Stored user attachment references and request metadata
reasoning_content	Returned reasoning text or summary
reasoning_details_json	Structured reasoning data from supported adapters
tool_calls_json	Assistant function calls, including names, arguments, and IDs
tool_call_id	Links a tool result to its assistant call
citation_sources_json	Sources used for citation rendering
chat_id, created_at, deleted_at	Ownership relation, ordering, and soft deletion

The full schema is in Database Schema. Reasoning is stored separately so the UI can display it separately and the request builder can filter it without changing the saved answer.

In-Memory vs Database Representation

Agent.messages holds dictionaries. Tool calls, content parts, and structured reasoning are lists or objects there; their database counterparts use JSON text columns. ChatContextService in src/numi_chat/chat/context.py converts between them.

Runtime messages can also contain temporary data, such as prepared file context and database IDs used to track history boundaries. Those internal fields are not part of the provider protocol.

Message Serialization

The Message model in src/numi_chat/data/models.py provides JSON accessors for structured fields. Repositories save messages and the context service reconstructs them when loading history. JSON text storage does not mean that all fields are suitable for every model API; conversion happens separately.

The Message Preparation Filter

Before each model request, the agent:

1. Reconstructs permitted attachment content and adds current user context.
2. Removes internal fields and filters reasoning through prepare_messages().
3. Includes schemas only for the tools active in this chat.
4. Applies the context budget, omitting older complete turns if necessary.
5. Lets the selected adapter convert messages to its wire protocol.

prepare_messages() does not strip reasoning merely because a model uses effort mode. It checks provider behavior and preservation flags, with special handling for tool-call continuation. It also does not validate all tool arguments or remove arbitrary tool calls. See Preserved Reasoning for the exact replay distinction.

Chat History Management

History Loading

The context service loads the system message and recent non-deleted complete turns, bounded by PROMPT_MAX_HISTORY_MESSAGES and PROMPT_MAX_HISTORY_CHARS. It keeps user/assistant/tool groups together so a request does not begin with an orphan tool result. When older turns are omitted, the runtime adds a notice telling the model not to assume those details. If the latest turn alone exceeds the loading budget, loading fails with a context-budget error.

A valid compaction checkpoint contributes a summary and the boundary through which history was summarized. Later original messages are loaded after it. The checkpoint changes active model context; it does not delete the original transcript. See Configure Auto-Compaction.

Caching

A ChatContextService instance caches chat metadata and loaded history. Saving a message or compaction checkpoint invalidates its history cache; refreshing chat settings invalidates its metadata cache. These caches are local to the service instance.

WebSocket Streaming

The connection manager broadcasts JSON events and assigns event_id and turn_id values. These are application WebSocket messages, not Server-Sent Events.

Event	Meaning
turn_start	A turn producer has started, with a history boundary
accepted	The submitted user message has been saved
reasoning, content	Incremental model output
tool_preparing	A tool call is being assembled from the model stream
tool_start, tool_end	Tool invocation and completed result
done	A final assistant response was saved without further tool calls
error	The turn encountered a failure, cancellation, or limit
turn_end	The producer has finished this turn
title_update	An optional title update after a successful final answer

This table describes the main flow; API Reference describes request and event payloads.

The browser can reconnect to an active turn using its turn and event IDs. The server replays retained events or a coalesced snapshot from bounded memory. Disconnecting the browser does not itself cancel the producer. Those buffers are temporary; after a server restart, the database supplies the persisted history instead.

Tool Message Flow

Each tool result is paired with a call by tool_call_id. For example, the internal history can contain:

[
  {
    "role": "assistant",
    "content": null,
    "tool_calls": [{
      "id": "call_123",
      "type": "function",
      "function": {"name": "example_tool", "arguments": "{}"}
    }]
  },
  {"role": "tool", "tool_call_id": "call_123", "content": "Example result"}
]

Here example_tool is illustrative. Actual tools are supplied by the active registry. Results can reach the UI in completion order while the stored history keeps their original call order. The adapter translates this pairing into Chat Completions tool messages, Responses function-call items, or Anthropic tool_use/tool_result blocks.

Cancellation and persistence limits

The runtime saves available partial assistant output when cancellation or a stream failure occurs after output has begun. It does not save every streamed token in its own transaction. An abrupt process failure can lose unsaved text, and a tool result may have appeared in the UI before it is committed.

Cancelling stops pending work and closes the model stream where possible. Already-running tool side effects are allowed to finish before cancellation is reported; a Python thread cannot safely undo an external action. These boundaries matter when deciding whether to retry a turn.

For the relevant implementation, start with services/chat_stream.py, agent/runtime.py, chat/context.py, and web/connection_manager.py under src/numi_chat/.