AI Agents Arena
Framework Picker

How AI Agent Costs Work

Why multi-step agent loops cost 4x to 15x more than single prompts—and the exact formula connecting step growth, retries, caching, and success rate to unit economics.

By [Author Name — Update in includes/config.php] Last reviewed 2026-11-12 Tool: AI Agent Cost Calculator

1. Stateless APIs Mean You Pay to Replay Context Every Step

A common budgeting mistake occurs when teams treat an AI agent task like a single API request. Suppose a support engineer tests a prompt in a playground: 1,800 system tokens, 1,200 tool-definition tokens, and a 600-token user question. At 3,600 input tokens and 350 output tokens, a single turn on a balanced model costs fractions of a cent.

Production agents rarely finish in one turn. Because major LLM APIs (OpenAI Chat/Responses, Anthropic Messages, Google Gemini generateContent) are stateless across tool invocations, your application must resend the entire conversation prefix plus every tool response on every subsequent step. If the agent makes five LLM calls to inspect an account, query a ledger, verify a shipping rule, and draft a reply—and each tool call returns 900 tokens—the billed input volume looks like an arithmetic series:

  • Step 1: 3,600 input tokens
  • Step 2: 3,600 + 900 = 4,500 input tokens
  • Step 3: 3,600 + 1,800 = 5,400 input tokens
  • Step 4: 3,600 + 2,700 = 6,300 input tokens
  • Step 5: 3,600 + 3,600 = 7,200 input tokens

Summing those five turns yields 27,000 input tokens—not 3,600. Doubling the number of steps from 5 to 10 does not double the input cost; it increases accumulated tool-result tokens by roughly 4.5x because the quadratic sum N × (N - 1) / 2 dominates as N rises. You can verify this curve directly using the SVG step chart inside the AI Agent Cost Calculator.

2. Output Tokens and Reasoning Tokens Carry the Steepest Multiplier

Across OpenAI, Anthropic, and Google rate cards, output tokens typically cost 4x to 8x more per million tokens than standard uncached input tokens. On Anthropic's Claude Sonnet 5.5 ($2.00/1M input vs. $10.00/1M output) or OpenAI's GPT-5 ($1.25/1M input vs. $10.00/1M output), generating verbose chain-of-thought or large uncompacted JSON payloads quickly becomes the second largest line item.

For explicit reasoning models—including OpenAI's o3 and o4-mini, Anthropic extended thinking modes, and Google Gemini thinking tiers—internal reasoning tokens are billed at the output token rate even when hidden from the end user. Always include average thinking token counts inside your per-step output estimate when modeling reasoning models.

3. Why Prompt Caching Changes Multi-Step Math

Prompt caching reverses part of the replay penalty by discounting repeated prefixes. According to official Anthropic pricing docs, OpenAI pricing docs, and Google Gemini pricing docs (verified 2026-11-12):

  • OpenAI: Cache hits on GPT-5 and GPT-4.1 tiers apply a 75% to 90% discount off standard input rates (for example, GPT-5 drops from $1.25/1M to $0.125/1M cached).
  • Anthropic: Cache reads cost 0.05x to 0.10x of base input (90% to 95% savings), while ephemeral cache writes incur a 1.25x (5-minute TTL) or 2x (1-hour TTL) initial write fee above minimum token thresholds.
  • Google Gemini: Implicit caching is enabled automatically on modern Gemini models (2.5 and newer) when prefix thresholds are met, passing through roughly 90% input savings without storage overhead.

In a 10-step agent run where the static system prompt, tool JSON schemas, and earlier turns stay stable at the front of the prompt, achieving a 70% to 80% cache-hit rate often cuts total LLM spend in half. Conversely, some open-weight serverless endpoints (such as Groq Llama 3.3 70B) do not publish a cached-input discount tier, which is why our calculator treats unverified cache rates as standard input pricing.

4. Retries, Escalations, and Success-Adjusted Unit Cost

Raw token math assumes every tool call succeeds on the first try and every task resolves cleanly. Production telemetry shows otherwise:

  1. Step Retries: Rate-limit 429s, transient API timeouts, and malformed tool-call JSON trigger automatic retries inside frameworks like LangGraph or CrewAI. An 8% to 15% retry overhead multiplies effective LLM calls by 1.08x to 1.15x.
  2. Failure & Human Escalation: If an agent fails on 10% of attempts after executing 6 steps, those wasted tokens still appear on your invoice. True unit cost per successful task equals Total Attempt Cost / Success Rate. At an 80% success rate, every successful resolution effectively carries a 1.25x cost multiplier.
  3. External Tool Fees: Paid search APIs, headless browser sessions, vector queries, and third-party enrichment APIs add fixed per-call charges outside the LLM bill.
  4. Tail-Risk Variance (P50 vs. P90): Average step counts mask skewed distributions where 10% of hard tasks trigger 2x to 3x more turns. Budgeting with a P90 variance reserve prevents surprise mid-month spend spikes.

Putting the Formula Into Practice

To test these dynamics against your own workload, open the AI Agent Cost Calculator or start from a domain baseline like the Customer Support Agent Cost Calculator or Coding Agent Cost Calculator. Once you have your baseline monthly figure, read How to Reduce AI Agent Costs and Tokens and Context Windows Explained to bring spend down systematically.

Primary Sources