Agent Inference Cost: Why One Call Becomes Twelve in Production

Every cost estimate for an agentic workflow starts the same way: tokens per call × model price × expected requests. That arithmetic is correct. It's also describing a system that doesn't exist in production.

A procurement approval agent doesn't make one inference call per user request. It classifies the request, extracts vendor and budget details, checks the approved vendor list, verifies department budget, routes to the right approval tier, drafts the approval message, parses the manager response, runs a compliance check, writes to the PO system, and notifies the requestor. Ten inference calls on the happy path. When the vendor lookup returns ambiguous results, add two more. Twelve calls is typical — and that's a well-scoped, single-purpose agent.

The same pattern appears everywhere: a code review agent working through diff analysis, context retrieval, security review, and comment synthesis makes 7–9 inference calls per PR. A customer support agent that retrieves history, classifies intent, drafts a response, and checks policy compliance makes 6–8. Multi-agent orchestration stacks more on top.

Call count alone doesn't explain the full cost gap. Agentic models require 5–30× more tokens per task than a standard chat interaction; multi-agent research pipelines push higher^[8]. That gap isn't model pricing — it's the inference multiplier: a compound of call count, context accumulation, tool schema overhead, and retry tax. Understanding all four components is what separates a realistic budget from a billing surprise.

One team running a LangChain four-agent research pipeline discovered this the hard way: the agents entered an infinite retry loop and ran for eleven days before anyone noticed. The bill was $47,000^[8].

Component 1: Call count. This is the part every team eventually notices. A workflow with 12 inference calls per user request costs at least 12× a single call — before anything else compounds. Most cost estimates stop here. The problem: call count is a multiplier on the three components below, not the final answer.

Component 2: Context accumulation. Every subsequent call in a multi-turn agent loop re-sends the full accumulated conversation history. Step 5 doesn't pay for step 5's reasoning — it pays for steps 1 through 5 in full, as input tokens. In a documented 5-step agent run, per-call input token counts grew: 888 → 3,400 → 8,900 → 14,200 → 18,900^[2]. The cost of step 5 alone exceeded steps 1 and 2 combined. The formula is triangular, not linear: total input tokens across N steps equals roughly N(N+1)/2 × average tokens per step. At 12 steps, the context accumulation factor is approximately 6.5× what a flat per-step estimate predicts.

Component 3: Tool schema overhead. Agent frameworks inject every registered tool's full schema into every inference call — whether or not the tool gets called. A single tool definition runs 550–1,400 tokens depending on description length and parameter detail^[3]. In multi-server MCP deployments, the problem scales fast: three MCP servers at 30 tools each means 90 tool definitions injected before a single character of user input. At 200 tokens per definition, that's 18,000 tokens of overhead per call — before any reasoning tokens^[9]. Scale to ten servers and the number climbs past 100,000 tokens per request. In static toolsets without dynamic loading, tool schema overhead accounts for 60–80% of total token usage^[1].

Component 4: Retry tax. Failed tool calls don't disappear. The error, the model's recovery attempt, and all intermediate reasoning accumulate in context and get re-sent on every subsequent step. Most agent frameworks — LangChain, LlamaIndex, the OpenAI and Anthropic tool-use SDKs — retry by replaying conversation history, not by retrying the single failed step in isolation^[8]. A 10% per-step failure rate, compounded across 10 steps without circuit breakers, multiplies costs several times over. The compounding is quadratic in context size, not linear in step count — that's the part practitioners miss. One team cut their per-task tool call count from 14 to 2 by adding explicit SUCCESS/FAILED terminal states to tool responses, eliminating retry loops that were consuming the majority of their budget^[2].

The structural reason cost estimates fail is that engineers price multi-step agent workflows as N independent inference calls. They're not independent. Each call carries the full history of everything that came before it, because that's how the API works.

For a leadership audience, the math does not need to live in a code-style box. The practical rule is simpler: every additional step carries the history of the steps before it, so the budget grows faster than the visible workflow count.

A 10-step workflow does not behave like 10 independent calls. The repeated context behaves closer to 55 units of accumulated history. A 12-step workflow behaves closer to 78 units. That is why the invoice feels disconnected from the initial estimate: the team priced the visible steps, not the accumulated history each step drags forward.

Apply this to a 12-step procurement agent on claude-sonnet-4-6 ($3/M input, $15/M output)^[6]. Reasonable assumptions: 3,000-token system prompt plus tool schemas, 200-token average user input per step, 500-token average tool result per step.

Naive estimate: 12 × (3,000 + 200 + 500) = 44,400 input tokens.

Formula estimate: N·S is 36,000; N·u is 2,400; N(N+1)/2 × r is 78 × 500 = 39,000. Total: approximately 77,400 input tokens — 1.74× the naive estimate before a single output token is counted.

Add output tokens, the retry path on vendor ambiguity, and tool schema overhead from a moderately sized MCP toolset, and the real multiplier on the naive budget reaches 3–5× for this single workflow. That's before any multi-agent orchestration layer.

This compounds further when you account for token price asymmetry. Output tokens cost 5× more than input tokens on Sonnet-class models. At high step counts, output token costs dominate — and unlike input tokens, output tokens don't benefit from prompt caching.

MCP's tool-injection model has a predictable failure mode at enterprise scale: every connected server contributes its full schema to every inference call, regardless of which tools the current step needs.

A GitHub MCP server with 30 tools, each averaging 550 tokens, contributes 16,500 tokens of overhead per call — before a single token of user input. Add a Jira server (20 tools × 800 tokens average = 16,000 tokens) and a Salesforce server (25 tools × 700 tokens = 17,500 tokens) and you have 50,000 tokens of schema overhead on every step of a 12-step workflow. Across the full workflow, that's 600,000 tokens of pure overhead — at Sonnet pricing, $1.80 per workflow run^[9].

This pattern explains why tool schema overhead can dwarf all other cost components in enterprise MCP deployments. The measured range in production multi-server setups is 10,000–100,000+ tokens per call^[1][9].

The fix is dynamic tool loading: classify the current step type first using a cheap model (Haiku 4.5 at $0.25/M), then inject only the tool subset relevant to that step class. A classification call costs a few hundred tokens; the savings on subsequent steps return that cost many times over. The GitHub MCP server that consumed 55,000 tokens per call in a static deployment drops to roughly 1,000 tokens per call with lazy-loaded schemas — a 55× reduction in schema overhead for that component alone^[9].

Most engineers model retry costs as a simple percentage uplift: 10% failure rate means 10% more cost. That's wrong. The error accumulates in context and gets re-sent on every subsequent step.

Here's the mechanics. Step 7 fails — the tool returns a malformed response. The agent appends the error to conversation history and retries. The retry itself costs step-7 context size × the full input token price. If it fails again, both the original error and the first retry error are now in context for retry 2. The cost of each retry grows with the number of prior failures.

Layer on retries at multiple levels — SDK retries inside a tool call, middleware retries wrapping the tool, agent-level retries wrapping the loop — and you hit multiplicative chains. Three retries per layer across five chained tools is the canonical retry storm: a single user request can produce up to 3^5 = 243 backend calls in the worst case^[8].

The four failure patterns that trigger the worst retry storms:

1. Absent terminal states. When tool responses don't include explicit SUCCESS/FAILED signals, agents re-invoke the tool trying to confirm completion. Adding terminal states cut one team's tool calls from 14 to 2 per task^[2].
2. Ambiguous tool schemas. Vague parameter descriptions cause models to guess wrong, producing invalid calls. The model then generates corrective reasoning that bloats context further.
3. Stateful tools without idempotency. Tools that fail partway through an operation leave the world in a state the agent doesn't understand. Recovery reasoning is expensive.
4. Multi-level retry without coordination. SDK-level retries inside a tool can generate multiple successful side effects before the agent-level retry decides the whole step failed — then re-triggers the tool, doubling the side effects and the cost.

Budget protection from retry storms requires circuit breakers at the workflow level, not just the call level. A per-workflow token budget ceiling that triggers graceful exit is the floor; an escalation path that pages the on-call before the budget is exhausted is the ceiling.

The only way to know your workflow's actual inference multiplier is to instrument it and measure from staging runs. A spreadsheet model can get the formula right; it cannot know your tool schema sizes, your retry rates, or your actual context accumulation slope.

The measurement pattern: wrap every inference call inside a parent workflow trace span, record per-call input and output token counts, track cumulative context size at each step boundary, and compute the ratio of total measured token spend to your step-1 token count × step count. That ratio is your multiplier.

OpenTelemetry's GenAI semantic conventions (gen_ai.usage.input_tokens, gen_ai.usage.output_tokens) give you per-call token data at the instrumentation layer^[7]. As of March 2026, most GenAI conventions are in experimental status, but the token usage attributes are stable enough for production cost tracking. The gap in most observability setups: teams see per-call cost but not per-user-action cost. Aggregating across the full workflow under a parent span closes that gap.

Run at least 20 staging executions — not 3. The distribution of per-workflow cost is heavy-tailed; a 3-run benchmark will almost certainly miss the retry paths that drive P95 spend.

For teams using Python, the same pattern maps directly to the Anthropic SDK:

Understanding the four-component multiplier tells you where to intervene. The highest-impact changes target components 2 and 3 — they compound against every call in the workflow.

Dynamic tool loading (targets Component 3 — tool schema overhead). The fastest win in most systems. Instead of injecting every registered tool's schema on every call, load only the tools relevant to the current step. In a GitHub MCP server deployment, switching from static to lazy-loaded schemas cut tool schema tokens from 55,000 to roughly 1,000 per initialization — a 55× reduction for that component^[9]. Implementation: classify the step type first with a cheap Haiku 4.5 call ($0.25/M), then load only the tool subset needed for that step class. The classification call costs a few hundred tokens; the savings across subsequent steps return that cost many times over.

Structured handoffs instead of full transcripts (targets Component 2 — context accumulation). Most agent loops pass full conversation transcripts between steps. Replacing those with structured JSON summaries — the decision made, the data retrieved, the constraints that apply — cuts context size per step by 50–70% while preserving what the next step actually needs^[11]. The formula doesn't change; the per-step r value in N(N+1)/2 × r does. Halving r on a 12-step workflow halves the accumulation component. The production consensus as of 2025–2026: anchored iterative summarization, where each compression updates a persistent JSON document rather than regenerating it from scratch, gives the most consistent results^[11].

Prompt caching on stable context (targets Components 2 and 3). System prompts and tool schemas are identical across calls within a workflow. Marking them with cache_control: { type: "ephemeral" } prices cache reads at 10% of standard input cost^[6]. A January 2026 multi-provider evaluation measured 41–80% cost reduction in long-horizon agentic tasks from prompt caching alone^[10]. One constraint: the Anthropic cache TTL is five minutes^[6]. Async approval flows and human-in-the-loop waits that span longer than five minutes re-incur the cache write cost on the next step — factor this into workflows with human review stages.

Combining dynamic tool loading and prompt caching addresses components 2 and 3 without touching workflow logic. Teams that implement both report 70–85% reduction from unoptimized baselines^[4].

The inference multiplier is the real unit of measurement for agent cost engineering. A 12-step workflow doesn't have a cost — it has a multiplier, and that multiplier interacts with model tier, tool schema size, and retry rate in ways a single-call benchmark can't capture.

Measure the multiplier from staging traces. Document it per workflow, at P95. Set the production budget ceiling against that number, not against the per-call estimate. The teams that avoid billing surprises share one discipline: they treat multiplier measurement as a launch gate, not a post-incident activity.