Prompt length is a weak objective. What matters is tokens per successful validated outcome, including retries and refusals. Short instructions can be cheap and vague. Detailed constraints can be expensive and clarifying. For Mistral Large 4 structured work, measure both styles on the same fixture pack before freezing a template.
What Token Efficiency Means
Cost follows tokens, not intentions
Providers bill input and output tokens. A prompt that feels clear but doubles input size must reduce failures or output length enough to compensate. Efficiency is an accounting identity, not a writing aesthetic. Track input, output, and retries under list prices you verify on the current rate card.
Success must be defined first
Define success as schema-valid and semantically accepted by your checkers. A short prompt that yields frequent invalid JSON is not efficient. A long prompt that always validates but answers the wrong task is not efficient either. Write the acceptance tests before comparing templates.
Steady templates beat clever one-offs
Compare fixed templates across a fixture suite. Ad lib rewrites between runs confound results. Version prompt templates like code. Pin the model id so preview drift is not mistaken for a template win.
Efficiency formula in practice
Compute average tokens in, tokens out, attempts, and dollars per accepted item. Hypothetical improvements should be replaced by measured ratios on your fixtures. Do not import another team’s “tokens saved” claim without their harness.
Short Instructions
Where short prompts help
Classification into a small enum, simple extraction with a tight schema, and tasks where the schema itself carries most constraints can thrive with brief instructions. The schema and validators do the heavy lifting. Extra prose may add cost without changing accept rates.
Failure modes of brevity
Short prompts may omit edge handling: empty inputs, conflicting fields, or refusal cases. The model may invent fields or ramble. If you choose brevity, invest in schema strictness and strong server-side checks. Brevity without validators pushes complexity into incidents.
Operational benefits
Smaller prompts are easier to review, cache, and audit. They reduce accidental leakage of large policy documents into every call. Teams can reason about behavior changes faster when the template fits on one screen. That human efficiency sometimes matters as much as token efficiency.
When to stop shortening
If semantic accept rates fall or retries climb, you have overshot. Restoring a few constraint sentences is cheaper than building elaborate repair loops. Watch tokens per success, not tokens per attempt, when deciding.
Detailed Constraints
Where detail helps
Complex structured extraction, multi-field dependency rules, and tool-calling catalogs often need explicit constraints. Detail can reduce illegal tool names and missing required fields. It is most useful when constraints match what validators will enforce, teaching the model the shape of success.
Failure modes of verbosity
Long policy text can dilute attention, raise cost, and still be ignored under injection. Duplicating the entire schema in natural language and in JSON Schema may waste tokens. Prefer machine-checked schema as authority and keep natural language for guidance that schema cannot express.
Maintenance cost
Detailed prompts drift as products change. Stale constraints cause systematic errors that look like model regressions. Own prompt templates in version control with review. Treat a prompt edit like a code edit with fixtures attached.
Detail without duplication
Reference field names consistent with the schema. Avoid restating every type in prose. Use short examples only when they are accurate and licensed for use. Bad examples are expensive twice: tokens paid and behavior taught incorrectly.
Measurement Harness
A/B templates on one suite
Run short and detailed templates on the same fixtures, same model id, same temperature, and same validators. Record attempts, schema pass, semantic pass, and token totals. Reject harnesses that change more than one variable per comparison.
Include retries in the bill
If the client retries on timeout, include those tokens. If you re-ask after schema failure, include those tokens too. Efficiency that ignores retries is fiction. Prefer fixing the template or schema to building automatic repair storms.
Output length pressure
Detailed constraints sometimes reduce chatter and sometimes increase explanatory padding. Measure output tokens separately. If product does not need explanations, instruct the model to return only the schema object and enforce that in validation.
Results table template
Template | Input tok avg | Output tok avg | Semantic pass | Tok per success |
|---|---|---|---|---|
Short A | measure | measure | measure | compute |
Detailed B | measure | measure | measure | compute |
Hybrid C | measure | measure | measure | compute |
Replace measure and compute with your run data; do not invent numbers for publication.
Decision Patterns
Schema-first hybrid
Often the efficient pattern is a short instruction plus a strict schema and solid validators. Detail goes into machine-checked structure, not paragraphs. Use extra prose only for rules you cannot encode yet. This pattern aligns with production boundaries: the application still decides correctness.
Progressive enrichment
Start short. Add constraint sentences only for fixture categories that fail. Expand the template using evidence, not preference. This keeps the prompt close to the minimum that meets accept rates on your distribution.
Route by task type
Simple routes keep short templates. Complex routes load detailed ones. A single global mega-prompt is rarely optimal. Gateways can select templates by route id, which also clarifies cost attribution per product surface.
Pattern comparison
Approach | Token profile | Risk | Fits |
|---|---|---|---|
Short only | Low input | Higher invalid rates | Simple enums, tight schema |
Detailed only | High input | Drift, dilution | Complex dependencies |
Hybrid schema-first | Moderate | Needs schema work | Most production extraction |
Per-route templates | Variable | More ops surface | Multi-product gateways |
Pitfalls and Limits
Caching interactions
If the provider offers cached input pricing, stable prefixes can change economics for long shared instructions. Verify current cache pricing and behavior before designing around it. Do not assume cache hits in benchmarks without measuring them.
Multimodal tokens
Images change token accounting. A short text instruction with large images is not a short request. Cap image counts and resolutions in the harness and in production. Compare templates with the same visual inputs or you will misattribute cost.
Preview model changes
Mistral Large 4 preview behavior may shift. Re-run the harness when pins change. A detailed template that won last week might lose after a checkpoint update. Efficiency rankings are pinned results, not permanent laws.
Human readability versus machine need
Sometimes a slightly longer prompt is kept because on-call engineers must understand it at three in the morning. Document that choice. Token efficiency is a goal, not the only goal, in operational systems.
Production Verdict
Efficiency rule
Optimize tokens per accepted result under a pinned model and fixed validators. Short is not always cheaper. Detailed is not always clearer. Evidence from your suite decides.
Checklist
Success criteria written. Templates versioned. Model id pinned. Input and output tokens logged. Retries included. Schema authority clear. Per-route templates considered. Harness scheduled. Prices verified against the current rate card. Multimodal costs included when relevant.
Practical default
Begin with schema-first short instructions. Add the smallest constraint text that lifts semantic pass rates on failing cohorts. Re-measure after each addition. Stop when tokens per success stop improving.
Boundary
Prompt text proposes behavior. Validators and application rules enforce it. Token efficiency work that weakens validation to “save tokens” is not efficiency; it is deferred incident cost. Make the model useful. Keep success definitions deterministic and measured.
