Python workers are common; Rust workers fit when you want strict parsing and tight control over async tasks. A tool-calling worker pulls jobs, asks Mistral Large 4 for the next action, validates tool names and arguments, executes only allowlisted tools, and writes results. The model proposes calls. The worker decides execution.
Worker Shape
Job intake
Jobs arrive from a queue with an idempotency key, tenant scope, goal payload, and budget counters. The worker claims a job with a lease. On success it acks; on retryable failure it releases with backoff. At-least-once delivery demands idempotent tool effects or carefully recorded side-effect markers.
Runtime
Tokio provides timers, HTTP to Mistral, and concurrent tool execution when policy allows. Prefer structured concurrency: spawn tool tasks under a budget, cancel on deadline, and never detach work that can mutate state without tracking. A runaway task is a production incident.
State machine
States include claimed, model_called, tools_validated, tools_executed, completed, and failed. Persist transitions for recovery. If a process dies after tool execution but before ack, restart must observe the side-effect record and not double-apply. Design that path before enabling mutating tools.
Privileged components
HTTP client keys, tool credentials, and queue credentials load from the environment at startup. Validation modules still avoid network side effects. Keep a hard line between the Serde admission path and the tool execution path even inside one binary.
Talking to Mistral Large 4
Request construction
Build chat or tool-capable requests with explicit model id, temperature, and max tokens. Include tool definitions that match your allowlist, not an open catalog of every internal API. Smaller catalogs reduce inventiveness and cost. Pin the model id used in production.
Response handling
Read the full body under a request deadline. Stream only if you have a designed partial parser; otherwise buffer with a size cap. Extract tool call entries into raw JSON values ready for Serde. Missing tool fields are validation failures, not opportunities to invent names.
Transport retries
Retry transient HTTP failures with jittered backoff and a hard attempt cap. Do not retry on 4xx validation-like errors from your own logic. Count attempts against the job budget so a bad night of 429s cannot infinite-loop a worker fleet.
Observability
Log request id, job key, attempt, latency, and token usage when provided. Avoid logging full prompts if they may contain secrets. Metrics should separate provider errors from tool execution errors so on-call knows which lever to pull.
Serde Validation of Tool Calls
Allowlist first
Check tool name against a static map before parsing arguments. Unknown names fail the job step. Dynamic tool registration at runtime without review undermines the security model. Ship new tools as code with schema structs and tests.
Argument structs
Each tool has a Serde struct with deny-unknown fields where appropriate. Enums constrain modes. Numeric ranges can be enforced post-parse in pure functions. Reject rather than clamp when clamping would change user intent on money or destructive operations.
Batch policy
If multiple tool calls arrive, validate all before executing any mutating tool. Partial validation with interleaved execution creates partial states. For read-only tools, parallel execution under Tokio can be acceptable after validation of the whole batch.
Validation outcomes table
Case | Result | Execute tools? |
|---|---|---|
Unknown tool name | Error | No |
Known tool, bad args | Error | No |
All valid read-only | Ok | Yes, maybe parallel |
All valid mutating | Ok | Yes, ordered, idempotent |
Mixed valid/invalid | Error by default | No |
Executing Tools Under Tokio
Timeouts per tool
Every tool future gets a timeout. Nested calls cannot inherit unbounded time from the parent job. When a tool times out, mark failure, record partial outcomes if any, and follow compensation policy. Silence after timeout is not success.
Concurrency limits
Use semaphores to cap concurrent outbound calls. A fleet of workers each spawning unbounded tasks can overwhelm downstream systems. Configure limits from environment and load-test them. Prefer fairness across jobs over maximizing single-job parallelism.
Idempotency tokens
Pass job key and step index into tools that mutate. Tools should store completion markers transactionally with the effect. Without this, queue redelivery duplicates emails, charges, or tickets. Read-only tools can skip markers but still need timeouts.
Cancellation
When the job deadline fires, cancel outstanding tasks and avoid starting new ones. Cancellation should be tested; some HTTP clients need explicit abort. Dropping a future without abort may leave work running and still costing money on external APIs.
Failure Modes Specific to Tool Calling
Hallucinated tools
Models invent plausible internal API names. Allowlists stop execution. Track denial counts; spikes may indicate prompt drift or catalog mismatch after a deploy that renamed tools without updating definitions sent to the model.
Argument injection
Untrusted goal text may try to embed extra JSON fields. Deny-unknown and typed parse reduce risk. Still treat string fields as data, not as code paths. Do not eval model strings. Do not build shell commands by concatenating model text.
Retry amplification
Provider retries plus tool retries plus queue retries multiply load. Assign budgets at each layer with clear ownership. Document which layer retries what. Integration tests should prove a single logical failure cannot trigger unbounded attempts.
Comparison of retry layers
Layer | Retries | Not retried |
|---|---|---|
HTTP to Mistral | Transport, 429, 5xx | Your validation errors |
Tool HTTP | Tool-specific transient | Auth errors, 4xx business |
Queue | Lease expiry, worker crash | Permanent validation failure |
Testing and Deployment
Unit tests for Serde
Feed illegal argument JSON for each tool. Assert errors. Feed legal JSON and assert struct equality. These tests are cheap and should outnumber live model tests. Include unicode, empty strings, and boundary numbers.
Integration tests with recorded responses
Store sanitized Mistral responses containing tool calls. Run the worker against a mock HTTP server. Assert which tools ran and which were denied. Live model calls belong in occasional manual packs, not in every CI run.
Load and soak
Run workers against mock providers to verify semaphore behavior and queue claim logic. Watch for task leaks under cancellation. Soak tests catch timer and connection pool issues unit tests miss. Fix leaks before enabling real credentials.
Deployment checklist items
Pin model id and tool catalog version together. Provide runbooks for upstream 429 storms. Expose metrics for denials, tool timeouts, and job completion. Start with read-only tools in production, then enable mutations behind flags.
Production Verdict
Worker rule
Serde admits tool arguments. Tokio bounds execution. Allowlists define capability. Mistral Large 4 does not receive direct access to privileged clients. That arrangement keeps agent-like behavior inside engineering controls you can review.
Checklist
Idempotent jobs. Leased claims. Pinned model id. Tool catalog versioned. Deny-unknown arguments. Per-tool timeouts. Concurrency caps. Layered retry budgets. Fixtures for denials. Metrics for each failure class. Ordered rollout of mutating tools.
When to prefer Python workers
If the team lacks Rust experience and load is modest, a Python worker with the same allowlist discipline is valid. Rust becomes worthwhile for strict parsing, performance under heavy validation, or shared binary policy across services. Pattern matters more than language fashion.
Boundary
Tool calls are proposals. Execution is a privileged, timed, idempotent act. Make the model useful for proposing the next step. Keep the system deterministic when deciding whether that step runs, how long it may run, and how many times it may try.
