AI Agents Arena
Framework Picker

Cost Model & Verification Methodology

Exact mathematical formulas, caching and retry rules, P50/P90 variance assumptions, and official pricing verification policies behind AI Agents Arena.

Pricing & specs verified 2026-11-12 Open Cost Calculator Open Framework Picker

1. Exact Agent Cost Calculation Formula

Both our server-rendered PHP engine and client-side JavaScript engine execute the exact same deterministic sequence so default examples, shared query-string URLs, and live interactive updates always match.

11-Step Deterministic Cost Pipeline
PHP & Browser JS Parity
Stage 1 · Context & Token Growth
  1. Base Context tokens
    System Prompt + Tool Definitions + User Task
  2. Step i Input Tokens grows each tool turn
    Base Context + (i − 1) × Tool-Result Tokens Added per Step
  3. Peak Context checked against model context window
    Step N Input Tokens + Output Tokens per Step
Stage 2 · Prompt Caching & Per-Step LLM Cost
  1. Cached Tokens(i) when Prompt Caching On & rate published
    Step i Input × (Cache-Hit % / 100)
  2. Uncached Tokens(i) billed at standard input rate
    Step i Input − Cached Tokens(i)
  3. Step LLM Cost(i) rates per 1M tokens
    (Uncached(i) / 1M) × R_in + (Cached(i) / 1M) × R_cache + (Output / 1M) × R_out
Stage 3 · Retries, External APIs & Success Yield
  1. LLM Cost with Retries transient rate-limit & tool retries
    (Σ Step LLM Cost(i)) × (1 + Retry Rate % / 100)
  2. Attempt Cost single run incl. paid tool calls
    LLM Cost with Retries + (External API Calls × External API Cost/Call)
  3. Cost per Successful Task (P50) absorbs failed or escalated runs
    Attempt Cost / (Task Success Rate % / 100)
Stage 4 · Variance Reserve & Budget Horizon
  1. Budgeted Cost per Successful Task 1.00 for P50 · 1.35 for P90
    P50 Cost × Variance Multiplier
  2. Daily, Monthly & Annual Spend 30-day month · 365-day year
    Daily = Budgeted Cost × Tasks/Day  |  Monthly = Daily × 30  |  Annual = Daily × 365
Questions or rate-card corrections? Email contact@aiagentsarena.com Launch Cost Calculator →

Prompt Caching & Unpublished Cache Rates

When Prompt Caching is toggled on, the cached fraction of each step's input tokens is billed at the model's published cache-read rate (cached_input_per_1m). Several open-weight serverless endpoints—such as Groq Llama 3.3 70B Versatile, Groq GPT-OSS 120B, and Together AI Llama 3.3 70B—do not publish a public cache-read discount rate. In those cases, our catalog records null ("Not published"), and the engine conservatively bills 100% of input tokens at the standard input rate rather than guessing a discount.

Model Routing Blend

When Model Routing is enabled with a cheap-model share s = cheap_model_share_pct / 100, the calculator computes blended per-million rates: R_eff = (1 - s) × R_primary + s × R_cheap for input, cached input, and output tokens. Both the primary and secondary models must accommodate the peak context window for the workflow to be flagged as fitting cleanly.

P50 vs. P90 Variance Budgeting

Agent loop lengths are right-skewed: most requests finish near the median, while edge-case tasks trigger extra recovery steps. Our P50 setting uses a 1.00x multiplier for median expected spend, while P90 applies a 1.35x planning buffer for finance and engineering capacity budgeting.

2. Model Pricing Verification Policy

We never invent or extrapolate model token rates. Every entry in our catalog is verified directly against official vendor pricing documentation and stamped with a verified_on date (currently 2026-11-12). Where a provider uses tiered long-context pricing—such as Anthropic Claude Haiku 5.5 above 100K tokens or Google Gemini 2.5 Pro above 200K tokens—we model the standard baseline tier and disclose the higher long-context threshold in the model notes and in Tokens and Context Windows Explained.

Official Pricing Sources Checked (2026-11-12)

3. Worked Verification Example (Hand-Checkable)

To verify parity between our server-rendered PHP engine, client-side JavaScript calculator, and your own spreadsheets, consider a 3-step workflow on GPT-5 mini ($0.25/1M input, $0.025/1M cached input, $2.00/1M output, 400,000 context window) with 1,000 system tokens, 1,000 tool-definition tokens, 500 user tokens (base_context = 2,500), 300 output tokens per step, and 500 tool-result tokens added per step:

  • Step 1: 2,500 input tokens, 300 output tokens.
  • Step 2: 3,000 input tokens, 300 output tokens.
  • Step 3: 3,500 input tokens, 300 output tokens.
  • Peak Context: Step 3 input (3,500) + output (300) = 3,800 tokens (fits inside 400,000).
  • With 80% Prompt Caching: Total base input is 9,000 tokens (7,200 cached at $0.025/1M = $0.000180; 1,800 uncached at $0.25/1M = $0.000450) plus 900 output tokens at $2.00/1M ($0.001800) = $0.002430 base LLM cost.
  • With 10% Retries, $0.005 External API, 90% Success, P50: LLM with retries = $0.002673; Attempt cost = $0.007673; Cost per successful task = $0.007673 / 0.90 = $0.008526.

4. Comparison Table & Export Behavior

The full model comparison table evaluates all 27 catalog entries in single-model mode (with cheap-model routing turned off) so users can inspect an apples-to-apples ranking sorted ascending by projected monthly spend. Models whose context window is smaller than Peak Context remain visible for transparency but are clearly flagged as "Too small," while the lowest-cost qualifying model is highlighted as "Cheapest fit." Both CSV exports and shareable query-string links encode your active inputs locally in the browser without storing anything on our servers.

5. Framework Picker Scoring Transparency

Our Agent Framework Picker avoids opaque "AI scoreboards" and synthetic benchmarks. Each of the 8 options starts from an equal 20-point baseline and receives explicit additive or subtractive points per answer based on documented attributes: supported languages, cyclic vs. linear graph support, model-provider coupling, durable checkpointing, realtime voice primitives, and dependency weight. The tool displays every framework's final score alongside its official documentation link.

6. Update Cadence & Corrections

We review major LLM pricing pages and agent SDK repositories monthly and whenever providers announce tier revisions. If you spot a stale rate card, a deprecated framework release, or a newly published cache tier, please notify us via our Contact Us page so we can verify against primary documentation and update the catalog promptly.