How to Reduce AI Agent Costs

1. Stabilize Prompt Prefixes to Unlock 70%–90% Cache Hits

Prompt caching is usually the highest-return optimization because it requires zero model downgrade. Providers match cached prefixes strictly from the beginning of the token sequence. If you inject dynamic timestamps, random request IDs, or per-user profile JSON at the top of your system prompt, every single call misses the cache.

Structure messages in strict stability order:

1. Static System Instructions: Role, policy rules, and output formatting guidelines first.
2. Static Tool / MCP Definitions: Keep tool names, descriptions, and JSON schemas sorted deterministically and never mutate them per request.
3. Append-Only History: Never rewrite or reformat earlier turns mid-task unless you are performing an intentional context compaction reset.

Across OpenAI GPT-5, Anthropic Claude 4.5/5.5, and Google Gemini 2.5/3.x models, cached input reads drop by 75% to 95%. Test toggling Prompt Caching on and off inside the AI Agent Cost Calculator to measure the exact monthly delta for your step count.

2. Route Routine Steps to Economy Models

Not every step in an agent trajectory requires frontier reasoning. In a 10-step research or coding workflow, up to half of the LLM calls perform mechanical sub-tasks: classifying an incoming ticket, rewriting a search query, extracting three JSON fields from an HTML snippet, or summarizing a compiler log.

Routing 40% to 60% of those steps to economy models such as gpt-5-nano ($0.05/1M in, $0.40/1M out), claude-haiku-5-5 ($0.10/1M in, $0.50/1M out under 100K), gemini-3.5-flash-lite ($0.30/1M in, $2.50/1M out), or open-weight serverless endpoints like DeepSeek V4.1 Flash ($0.30/1M in, $1.20/1M out) slashes blended spend while keeping complex planning on your primary model. Try the Enable Model Routing toggle inside our Research Agent Cost Calculator or Coding Agent Cost Calculator.

3. Trim Tool Payloads Before Appending to History

Because tool results added at Step 1 are replayed at Step 2, Step 3, and Step 10, every wasted 1,000 tokens in an early tool output actually costs you 1,000 × (N - 1) input tokens across the remainder of the run.

• Filter JSON Responses Server-Side: Strip null fields, audit metadata, CSS/HTML wrappers, and unused nested arrays inside your tool wrapper before returning text to the model.
• Truncate Verbose Logs: In coding and DevOps agents, cap stack traces and test runner logs to the final 80–120 lines plus error summary lines, offering a separate read_log_range tool if the agent truly needs earlier lines.
• Return Handles Instead of Full Documents: Store large retrieved files in an artifact store and return a compact 200-token summary + handle ID rather than pasting 15,000 raw tokens into chat state.

4. Prune Unused Tool Schemas

Teams frequently register 30 to 50 tools (or attach multiple broad MCP servers) into every agent call. At 120 to 250 tokens per JSON function definition, 40 tools consume 5,000 to 10,000 tokens on every single step—even when the agent only uses two of them. Group tools by intent or stage and expose only the 3 to 8 tools relevant to the active workflow phase.

5. Enforce Step Caps, Strict Schema Validation, and Batch Tiers

• Hard Step & Budget Caps: Guard every loop with explicit max_steps and per-task token ceilings so runaway loops escalate cleanly instead of burning 40 turns.
• Constrain Tool Arguments: Use native Structured Outputs / strict JSON schemas to prevent malformed tool calls that trigger expensive retry turns.
• Use Batch APIs for Asynchronous Work: OpenAI, Anthropic, Google Gemini, Together AI, and Fireworks AI all offer roughly 50% discounts for asynchronous Batch API jobs. Nightly document enrichment, ticket QA auditing, and catalog tagging should never run on synchronous real-time endpoints.

Worked Before-and-After Optimization Example

Consider a 10-step research workflow processing 500 tasks per day on a frontier-class model ($2.00/1M input, $0.10/1M cached input, $10.00/1M output) with 2,500 system tokens, 2,000 tool-schema tokens, 800 user tokens, and 2,400 tool-result tokens appended per step:

• Unoptimized Baseline (0% cache, single frontier model, verbose 2,400-token tool payloads, 15% retries): Peak context reaches 26,900 tokens, total uncached input across 10 steps exceeds 161,000 tokens per attempt, and monthly spend reaches several thousand dollars once retries and 88% task completion are factored in.
• After Optimization (static prefix yielding 80% cache hit, tool summaries trimmed to 900 tokens, 50% routine steps routed to an economy tier, strict schema validation dropping retries to 5%): Input token volume falls by more than 60%, cached tokens bill at 5% to 10% of standard rates, and blended monthly spend commonly drops by 70% or more while latency improves because smaller payloads decode faster.

What Not to Optimize First

Avoid two common false economies. First, do not truncate essential error diagnostics or schema constraints to save a few hundred tokens if doing so lowers your task success rate from 94% to 78%—failed attempts still pay for LLM calls and external APIs while generating customer escalations. Second, do not split a clean single-agent tool loop into four separate "persona" agents unless parallel isolation is strictly required; inter-agent handoffs replay context across separate prefix boundaries and often reduce prompt cache hit rates.

Validate Changes Before Shipping

Before deploying prompt or routing changes, benchmark step count and completion rate together. Model both variables simultaneously in the AI Agent Cost Calculator, review How AI Agent Costs Work, and inspect Tokens and Context Windows Explained for schema sizing tips.