Skip to content

Tool System Design

Tools let the agent search the web, inspect chat attachments, execute sandboxed Python, generate files and images, and interact with configured external services. Before any tool can run, it must be discovered at startup, permitted by server policy and user tier, and active for the current chat.

For built-in tool schemas and parameters, see Tools Reference. For operator configuration, see Secure Tool Access.

Tool Architecture Overview

src/numi_chat/tools/*.py
    -> ToolRegistry: startup allowlist/denylist and is_available() checks
    -> Agent: core tools + chat-enabled optional groups (filtered by user tier)
    -> Wire Adapter: function schemas formatted for Chat, Responses, or Messages API
    -> ToolExecutorService: argument validation, parallel/serial execution, output caps
    -> Agent: tool result messages (with system_hint and usage) appended in call order

Tool Discovery and BaseTool (tools/base.py, tools/registry.py)

At startup, ToolRegistry imports every .py module in src/numi_chat/tools/ and instantiates concrete BaseTool subclasses that pass TOOLS_ALLOWLIST, TOOLS_DENYLIST, and is_available().

BaseTool Member Purpose
name, description, parameters Function name, description, and JSON Schema sent to the model
execute(**kwargs) Synchronous tool implementation returning a JSON string
is_available(), availability_reason() Checks required API keys or binaries at registration time
concurrency_safe True (default) if calls can run concurrently in a parallel batch; False for serial execution
_ok(), _error(), _noop() Helpers that serialize {"status": "success" \| "error" \| "noop", ...} JSON responses

Skills Integration (services/skills.py)

services/skills.py loads built-in skills (src/numi_chat/skills/) and private operator skills (state/skills/):

  1. The opening user message includes a compact <available_skills> index listing each active skill's name and description.
  2. When a task matches a skill, the model calls read_skill to read SKILL.md or reference files inside that skill folder.
  3. When Python code in code_interpreter references /mnt/skills/<name>, the executor verifies that the user's tier grants the Skills capability, mounts the skill directory read-only inside the sandbox, and preloads its allowlisted Python packages.

New public share snapshots strip read_skill calls and outputs automatically. See Manage Skills.

Execution Pipeline (services/tools/executor.py)

1. Active-tool verification and context injection

Before running a tool call, ToolExecutorService verifies that the tool name is in the chat's active tool set (blocking hallucinated or disabled tools) and injects server-owned chat_id, chat_uuid, and user_id values so model-supplied arguments cannot override them.

2. Forgiving argument validation

Arguments are validated against the tool's JSON Schema before execute() is called:

  • Common lossless model serialization quirks are coerced automatically: a JSON array passed as a string, a bare string passed where an array of strings is expected, or numbers/booleans passed as strings.
  • If a required argument is missing or has an incompatible type, the executor returns a structured error naming the bad field, suggesting the closest valid parameter name on typos, and listing the expected schema so the model can self-correct on its next step.
  • Extra unknown keys on an otherwise valid call are ignored with a note rather than failing the call.

3. Parallel vs. serial batching

Within a single model step, ToolExecutorService splits requested tool calls by concurrency_safe:

  • Concurrency-safe tools run in parallel on a bounded ThreadPoolExecutor, streaming tool_end events to the browser as each call completes.
  • Non-concurrent tools (such as code_interpreter, codemode, research_spawn, and research_poll) run one at a time after the parallel batch finishes.
  • Regardless of completion order, Agent.run_tools() appends the resulting role="tool" messages in the exact order the model requested them so tool_call_id pairings stay valid across all providers.

4. Code Mode (tools/codemode.py)

When the model needs to chain several read-only tool calls in a script, codemode runs Python inside the sandbox and routes nested tools.<name>(...) calls back through ToolExecutorService (up to four concurrent calls at a time). Every nested call obeys the chat's active tool set and argument checks, while only the script's final stdout enters the conversation history.

5. Error isolation and output caps

A failed tool call returns a JSON error message for that tool_call_id without aborting sibling tool calls in the same batch. Outputs exceeding AGENT_MAX_TOOL_OUTPUT_CHARS (64,000 characters) are replaced with a bounded preview and error notice so a single huge response cannot blow out the context window.

Schema Compaction and Output System Hints

  • Schema compaction (TOOLS_SCHEMA_MODE=compact): Truncates long tool descriptions (TOOLS_SCHEMA_DESCRIPTION_MAX_LEN=160) and optionally strips parameter descriptions (TOOLS_SCHEMA_STRIP_PARAM_DESCRIPTIONS) when building tool schemas for the model.
  • Runtime system hints (@tool_system_hints): Tools can attach post-execution guidance to their JSON result (system_hint) keyed by action or mode (* fallback). Hints steer the model's next step only after a tool actually runs, keeping the base system prompt and tool schemas lean.

Key Source Files

Path under src/numi_chat/ Responsibility
tools/base.py BaseTool contract, schema generation, _ok/_error helpers, @tool_system_hints
tools/registry.py Startup module discovery, CORE_TOOL_NAMES, CUSTOM_TOOL_GROUPS, allowlist/denylist filtering
services/tools/executor.py Active-tool check, argument coercion/validation, parallel/serial execution, output limits
agent/runtime.py Active tool selection, WebSocket tool events, citation extraction, result persistence
tools/code_interpreter.py Monty and Deno/Pyodide sandbox execution and chat file provisioning
tools/codemode.py Sandboxed Python scripts that orchestrate SCRIPTABLE_TOOLS