If one component both decides the plan and performs the actions, a single bad completion can refund money, delete data, or loop on the same tool. Split the roles. Mistral Large 4 returns a structured plan inside a closed vocabulary. Application code validates that plan and runs only handlers it already trusts. The model never holds execution credentials.
Why the Split Exists
Planning is advisory text
A plan is a structured proposal: ordered steps, typed arguments, and optional rationale fields your schema allows. It is not a transaction. Treating model text as already-authorized work collapses review into generation. Keep planning pure: no database writes, no network side effects, and no imports of modules that carry privileged clients inside the planner process.
Execution is ordinary code
Handlers are normal functions with unit tests, typed inputs, and explicit return values. They re-read current state, check predicates, and write through your data layer. They do not call Mistral to “decide whether to continue.” Once a step is approved by validation, behavior should be reproducible for the same state snapshot and the same plan bytes.
Threats the split reduces
Prompt injection that invents tools, plan inflation that exceeds budget, and hallucinated identifiers are rejected before effects. Stale plans still need handler-side version checks, because the world can change after the snapshot. The split removes unbounded improvisation; it does not remove concurrency control or business rules.
What the split does not claim
Separation does not make Large 4 deterministic. It makes the set of possible actions finite and reviewable. You still measure plan-accept rates, handler failures, and cost per successful job. A clean architecture with unmeasured failures is only a tidy incident.
Plan Schema and Snapshot Design
Closed step vocabulary
Enumerate every legal step type in JSON Schema and mirror it in application types. Unknown types fail validation. Optional arguments must be explicit; additional properties should be forbidden. When product needs a new capability, add a schema version, a handler, and fixtures in one change set rather than widening the enum casually in production.
Snapshot contents
Send only fields required for planning: entity ids, statuses, amounts, and policy flags the planner may read. Omit secrets, raw credentials, and unrelated tenant data. Hash the snapshot and store the hash with the plan so you can explain later why a step looked valid at proposal time without retaining excess personal data.
Prompt discipline
System instructions should define the planner role and point at the schema. Avoid few-shot examples that demonstrate forbidden tools. Examples teach behavior; illegal examples teach illegal behavior. Keep temperature low for planning so repeated runs on the same snapshot are easier to compare during evaluation.
Token and size budgets
Cap plan length, step count, and max tokens on the completion. Truncated JSON is a failure, not a partial plan to salvage with regex. Bound snapshot size so planning cost stays predictable. Long context is not a substitute for retrieval that selects the few records that matter.
Validation Gates Before Effects
Structural validation
Decode JSON, apply schema checks, then construct typed models. Missing required fields, wrong types, and extra keys stop the flow. Return stable error codes to callers without echoing raw model text to end users. Structural failure should never open a database transaction.
Budget and policy validation
Count steps, estimate fan-out, and enforce monetary or rate ceilings declared in configuration. A legal enum value with an absurd quantity is still a reject. Policy limits belong in application configuration, not in free-text instructions the model can be talked out of.
Referential validation
Confirm entity identifiers exist in the caller’s tenant and that the caller may act on them. Fail the entire plan if any step references out-of-scope data. Partial application of a mixed plan creates hard-to-reconcile states and confuses support teams.
Semantic business rules
Encode rules schema cannot express: refunds cannot exceed capture, closed accounts cannot ship, mutually exclusive flags cannot both be true. Run pure functions on the typed plan plus fresh reads. Schema validity is necessary and still insufficient for correctness.
Deterministic Execution
Dispatch table integrity
Map each enum value to a handler at startup and assert the map covers the schema exactly. Missing handlers abort process start. After validation, lookup failures are programming errors, not runtime opportunities to ask the model for advice.
Handler contracts
Each handler receives typed arguments, a unit of work, and clock or id generators you control. It re-checks preconditions against current rows. It never interprets free-form natural language as code. Calculations that matter for money or inventory live in tested functions, not in regenerated model prose.
Transactions and compensation
Prefer a single transaction when all steps share one database. For external systems, define compensation or reconciliation before enabling the step type. “The model will notice and fix it” is not a recovery strategy. Record which steps committed so retries do not double-charge.
Idempotency keys
Execution requests carry keys from the orchestrator. Replays return the stored outcome without re-running handlers. Do not silently request a new plan on every transport retry unless a human or explicit policy says re-planning is allowed for that job type.
Failure Modes to Fixture
Illegal plans
Fixtures should include unknown tools, extra properties, over-budget step lists, and cross-tenant identifiers. Each must fail before dispatch with unchanged storage. These tests protect against future “be helpful and accept almost JSON” changes in client code.
Stale state
Execute a plan after another transaction changes row versions. Handlers must abort. This case is common in concurrent support tools and is frequently missing from demos that run on quiet databases.
Retry loops
If a step may repeat, enforce schema maxima and executor caps. Tests should fail if a handler loops internally without a bound. Model text that says “try again forever” must not outrank your counters.
Injection in goal text
Goals that attempt to override the schema still pass only as strings into a planner that can emit only enumerated steps. Keep a fixture that tries to coerce a disallowed tool and assert rejection. Parsing success is not authorization.
Observability and Cost
Separate spans
Trace planning latency and tokens on one span; execution latency and entity ids on another. Label plan rejections by class: schema, budget, reference, business rule. Rising schema failures suggest prompt or model drift. Rising business-rule failures suggest stale snapshots or unclear goals.
Cost attribution
Charge planning tokens even when plans reject. Unlimited re-plan on validation failure is a cost bug. Gate automatic re-plans behind explicit policy and budgets. Report cost per successful execution, not only per provider call.
Comparison of control layers
Layer | Rejects | Retryable | Owner |
|---|---|---|---|
JSON decode | Truncation, syntax | No | Client |
Schema | Unknown steps, types | No | Shared schema package |
Budget | Too many steps | No | Config service |
Business rules | Domain violations | No | Domain library |
Handler precondition | Stale versions | Sometimes after refresh | Executor |
Operational limits
Planning belongs on a deadline-controlled client path. Execution may be async for long workflows, but status must be queryable. Do not hide multi-minute tool chains behind a single synchronous HTTP request without a product-level wait design.
Production Verdict
Rule to keep
Mistral Large 4 proposes plans inside a vocabulary you enumerate. Your application validates structure, budget, and domain rules, then executes with credentials the model never receives. Features that require the model to “just call the API” are incomplete designs.
Checklist
Schema and handler sets match at startup. Snapshots omit secrets. Validation performs no writes. Handlers re-check versions. External steps have compensation stories. Execution is idempotent. Metrics separate plan and execute. Injection and over-budget fixtures run in CI.
Staging bar
Do not enable write handlers in production until fixtures and startup equality checks exist. A prompt that usually works is not a control plane. Pin model ids used for planning and re-run the pack when previews change.
Boundary
The model proposes. The application decides. If a step cannot be validated, it does not run. That single rule prevents most agent incidents that start as impressive demos and end as irreversible writes.
