Tool System Design¶
Tools let the agent search the web, inspect chat attachments, execute sandboxed Python, generate files and images, and interact with configured external services. Before any tool can run, it must be discovered at startup, permitted by server policy and user tier, and active for the current chat.
For built-in tool schemas and parameters, see Tools Reference. For operator configuration, see Secure Tool Access.
Tool Architecture Overview¶
src/numi_chat/tools/*.py
-> ToolRegistry: startup allowlist/denylist and is_available() checks
-> Agent: core tools + chat-enabled optional groups (filtered by user tier)
-> Wire Adapter: function schemas formatted for Chat, Responses, or Messages API
-> ToolExecutorService: argument validation, parallel/serial execution, output caps
-> Agent: tool result messages (with system_hint and usage) appended in call order
Tool Discovery and BaseTool (tools/base.py, tools/registry.py)¶
At startup, ToolRegistry imports every .py module in src/numi_chat/tools/
and instantiates concrete BaseTool subclasses that pass TOOLS_ALLOWLIST,
TOOLS_DENYLIST, and is_available().
BaseTool Member |
Purpose |
|---|---|
name, description, parameters |
Function name, description, and JSON Schema sent to the model |
execute(**kwargs) |
Synchronous tool implementation returning a JSON string |
is_available(), availability_reason() |
Checks required API keys or binaries at registration time |
concurrency_safe |
True (default) if calls can run concurrently in a parallel batch; False for serial execution |
_ok(), _error(), _noop() |
Helpers that serialize {"status": "success" \| "error" \| "noop", ...} JSON responses |
Skills Integration (services/skills.py)¶
services/skills.py loads built-in skills (src/numi_chat/skills/) and private
operator skills (state/skills/):
- The opening user message includes a compact
<available_skills>index listing each active skill'snameanddescription. - When a task matches a skill, the model calls
read_skillto readSKILL.mdor reference files inside that skill folder. - When Python code in
code_interpreterreferences/mnt/skills/<name>, the executor verifies that the user's tier grants the Skills capability, mounts the skill directory read-only inside the sandbox, and preloads its allowlisted Python packages.
New public share snapshots strip read_skill calls and outputs automatically. See Manage Skills.
Execution Pipeline (services/tools/executor.py)¶
1. Active-tool verification and context injection¶
Before running a tool call, ToolExecutorService verifies that the tool name is
in the chat's active tool set (blocking hallucinated or disabled tools) and
injects server-owned chat_id, chat_uuid, and user_id values so
model-supplied arguments cannot override them.
2. Forgiving argument validation¶
Arguments are validated against the tool's JSON Schema before execute() is
called:
- Common lossless model serialization quirks are coerced automatically: a JSON array passed as a string, a bare string passed where an array of strings is expected, or numbers/booleans passed as strings.
- If a required argument is missing or has an incompatible type, the executor returns a structured error naming the bad field, suggesting the closest valid parameter name on typos, and listing the expected schema so the model can self-correct on its next step.
- Extra unknown keys on an otherwise valid call are ignored with a note rather than failing the call.
3. Parallel vs. serial batching¶
Within a single model step, ToolExecutorService splits requested tool calls by
concurrency_safe:
- Concurrency-safe tools run in parallel on a bounded
ThreadPoolExecutor, streamingtool_endevents to the browser as each call completes. - Non-concurrent tools (such as
code_interpreter,codemode,research_spawn, andresearch_poll) run one at a time after the parallel batch finishes. - Regardless of completion order,
Agent.run_tools()appends the resultingrole="tool"messages in the exact order the model requested them sotool_call_idpairings stay valid across all providers.
4. Code Mode (tools/codemode.py)¶
When the model needs to chain several read-only tool calls in a script,
codemode runs Python inside the sandbox and routes nested tools.<name>(...)
calls back through ToolExecutorService (up to four concurrent calls at a
time). Every nested call obeys the chat's active tool set and argument checks,
while only the script's final stdout enters the conversation history.
5. Error isolation and output caps¶
A failed tool call returns a JSON error message for that tool_call_id without
aborting sibling tool calls in the same batch. Outputs exceeding
AGENT_MAX_TOOL_OUTPUT_CHARS (64,000 characters) are replaced with a bounded
preview and error notice so a single huge response cannot blow out the context
window.
Schema Compaction and Output System Hints¶
- Schema compaction (
TOOLS_SCHEMA_MODE=compact): Truncates long tool descriptions (TOOLS_SCHEMA_DESCRIPTION_MAX_LEN=160) and optionally strips parameter descriptions (TOOLS_SCHEMA_STRIP_PARAM_DESCRIPTIONS) when building tool schemas for the model. - Runtime system hints (
@tool_system_hints): Tools can attach post-execution guidance to their JSON result (system_hint) keyed by action or mode (*fallback). Hints steer the model's next step only after a tool actually runs, keeping the base system prompt and tool schemas lean.
Key Source Files¶
Path under src/numi_chat/ |
Responsibility |
|---|---|
tools/base.py |
BaseTool contract, schema generation, _ok/_error helpers, @tool_system_hints |
tools/registry.py |
Startup module discovery, CORE_TOOL_NAMES, CUSTOM_TOOL_GROUPS, allowlist/denylist filtering |
services/tools/executor.py |
Active-tool check, argument coercion/validation, parallel/serial execution, output limits |
agent/runtime.py |
Active tool selection, WebSocket tool events, citation extraction, result persistence |
tools/code_interpreter.py |
Monty and Deno/Pyodide sandbox execution and chat file provisioning |
tools/codemode.py |
Sandboxed Python scripts that orchestrate SCRIPTABLE_TOOLS |