Multi-Agent Cost Modeling: Why the Tail Burned $47K in 11 Days

Two LangChain agents burned $47,000 over eleven days in November 2025.^[1] One generated queries. The other validated responses. They locked into a handshake. Neither was malfunctioning — they did exactly what their prompts instructed. The validator kept rejecting the generator's output as incomplete. The generator kept trying again. Eleven days. The burn rate became visible when someone finally opened the billing dashboard.

Monitoring was in place. It did not stop the loop. Cloud cost anomaly detection aggregates spend over 24-48 hour windows — by the time a threshold fires, a session burning $0.045 per minute has been running for days.^[2] Monitoring that catches a runaway 36 hours in does not prevent $47,000 in damage. It documents it.

The real failure happened before any code ran. The team modeled expected agent cost as an average: typical tokens times model price equals expected session cost. That formula is correct for the 65-70% of runs that complete cleanly. It has no term for the 2-5% of runs where termination conditions fail and context grows without bound. In multi-agent systems, that small tail is where almost all your money goes.

More monitoring will not fix this. Tighter alerts will not fix this. The fix is upstream: model cost as a distribution, not a point estimate, and put enforcement synchronously in-process — checked before each API call — instead of asynchronously from outside it.

Standard API cost modeling treats requests as deterministic: hit the backend, return a response, pay a predictable amount. Agentic workflows do not behave that way. Tool calls per session are variable. Retry cycles are variable. Context length — which sets the cost of every subsequent call — grows on every iteration.

An ICLR 2026 study of agentic coding found that for identical task specifications, some runs consumed 10 times more tokens than others. There was no quality bonus. The expensive runs were less accurate on average.^[3] Same task, same agent, different inputs, 10x spread in cost. No mean compresses that into a useful budget.

Briefcase AI analyzed 1.4 million production LLM conversations. P95 cost ran 3-4x the median. The tail of massive conversations — about 9% of sessions — accounted for more than half of total spend.^[4] Model from the mean and you have silently excluded the expensive half of your bill.

A January 2026 empirical study ("Tokenomics") of a ChatDev multi-agent system running 30 software development tasks found that input tokens averaged 53.9% of all consumption — and iterative code review alone consumed 59.4% of the total token budget, dwarfing initial code generation.^[12] The refinement loop is the cost, not the primary task. That changes where you instrument and where you cap.

Three tiers describe where agent cost actually lands. Happy path is the cheap, common case: the task completes in the tool calls and turns you predicted. Iterative search is the middle: retries, multi-hops, refinement on partial results. Edge-case recovery is the tail: a termination condition fails, context explodes, or two agents lock into the handshake that generates five-figure bills. Budget from your P95. Enforce at 3x P95. The edge-case tier is what the third multiple is there to catch — and it is exactly what every average-based budget pretends does not exist.

A June 2026 arXiv paper cataloged 63 confirmed production budget-overrun incidents across 21 orchestration frameworks spanning 2023-2026, each backed by a quoted GitHub issue and (where reported) a documented dollar loss.^[11] The researchers organized them into eight failure clusters — not eight separate problems, but eight expressions of the same root condition: no in-process enforcement primitive.

The clusters map cleanly onto where teams typically install controls after their first incident. They install logging; the failure that hits them next is not the same loop. It's a different cluster.

Cluster 1: Incompatible termination conditions. Two agents, each with correct exit logic, whose conditions are mutually satisfiable only after infinite attempts. The $47K loop was this cluster. Neither agent was broken. Their combined contract was.

Cluster 2: Unbounded retry amplification. A retry policy with no cumulative cap. Three retries on failure sounds safe. Three retries per tool call, per agent, per orchestration step, compounding across an eight-step pipeline, is not. The error rate does not change; the cost multiplier does.

Cluster 3: Context accumulation without summarization. No compression between agent handoffs. Full conversation history forwarded at each boundary. Cost grows quadratically with turns — O(n²) — because every agent reads the entire prior thread, not just the delta. Frameworks including LangGraph now ship summarization nodes precisely because this pattern is the default failure mode in multi-turn pipelines.

Cluster 4: Tool response amplification. A tool that returns arbitrarily large outputs — a database dump, a full API response, a scraped webpage — injected into context without truncation. One tool call can add 50,000 tokens to the context window and make every subsequent call in the session expensive.

Cluster 5: Parallel fan-out with no aggregate cap. An orchestrator spawning N sub-agents with individual session limits but no pipeline-level ceiling. Each sub-agent's limit trips independently. The orchestrator's aggregate cost is N times the per-agent limit, minus any that hit their individual ceiling before the pipeline completes. Parallel agents multiply the per-agent cost variance.

Cluster 6: Reasoning loop accumulation. Extended chain-of-thought or self-critique loops that generate tokens without making progress. The model keeps reconsidering. Tokens accumulate. The task does not advance. This cluster is more common with reasoning-optimized models (o-series, DeepSeek-R1) where verbose internal reasoning is priced the same as output tokens.

Cluster 7: Memory retrieval amplification. A retrieval layer that returns too many chunks per query — either because similarity thresholds are too loose or because the top-k is too high — injecting excess context before the model has a chance to reason. Each retrieved chunk is paid for at full input token price, across every turn that re-sends the retrieval result.

Cluster 8: Phantom delegation. A budget is delegated to a sub-agent but the parent retains a reference. Both spend against it. Neither sees the other's draw. This cluster is what the affine-typed Rust mitigation in the same paper^[11] is designed to make a compile-time impossibility — making double-spending structurally unrepresentable rather than a runtime check.

Multi-agent systems carry a cost property single-agent systems do not: context multiplies at every handoff. When an orchestrator passes work to a research agent, then forwards the result to an analysis agent, the analysis agent does not receive the research output alone. It receives the accumulated context of the entire session — the original request, the orchestrator's planning steps, the research agent's full output — as its input.

A 3-agent sequential pipeline where each agent produces 500-token outputs does not consume 3 × (1,000 input + 500 output) = 4,500 tokens. It consumes roughly 1,000 + 1,500 + 2,000 = 4,500 input tokens across the three agents, before any output tokens at all — a 4.5× multiplier on inputs alone, before retry overhead enters the picture.^[5] Group-chat patterns are worse. Five agents, ten rounds, 300 tokens per message: 15,000 tokens in shared context before any work happens, because every agent reads the full message history on every turn.

Chatbot cost intuitions do not transfer. Anthropic's own research finds agents burn 4× more tokens than direct chat, and multi-agent systems burn 15× more.^[6] One team migrated a simple RAG chatbot to an agentic pipeline. Monthly inference spend jumped from $4,200 to $31,000 — same underlying tasks, different architecture.^[7]

The computational cost of attention scales quadratically with context length — O(n²) in sequence length, with constant-factor optimizations but no change to the growth rate.^[13] Double the context, quadruple the computation. That math shows up on your bill even if your per-token price dropped 80% this year.

The handoff format is a cost decision. Full conversation history is the most expensive option. A compressed structured summary — key facts, decisions made, constraints that must persist — is cheaper and often produces better downstream reasoning, because the receiving agent reads signal instead of noise. This is an interface design problem. Infrastructure cannot fix what the interface gets wrong.

Every team that hits a cost spiral reaches for the same fix first: better monitoring. The reflex is correct for visibility — you need to know what happened. It cannot solve prevention, because monitoring is structurally asynchronous.

Cloud cost alerts aggregate spend over 24-48 hour rolling windows. A session burning $10 per hour accumulates $240 before a daily anomaly threshold even has the data to fire on. That assumes a tightly-calibrated threshold. A team running legitimate large batch jobs alongside agents will eat persistent false positives, train its engineers to dismiss cost alerts as noise, and then miss the real event when it arrives. One engineer who traced this failure mode in a production multi-agent framework put it cleanly: 'Per-session accounting without a synchronous enforcement point tends to lag behind the actual spike. By the time you observe the overage, the burst has already happened.'^[9]

Prometheus-based dashboards have the same property. Even at one-minute scrape intervals, an alert rule must evaluate, match a condition, and route a notification before anyone can act. A zombie agent stuck in a reasoning loop burns $4-5 in a single query.^[7] Multiply by concurrent sessions and the 30-90 minutes it takes for an alert to reach someone with permission to kill the process. Monitoring is necessary. It is not sufficient.

The dashboard sees outputs. It cannot stop the process in flight. For that you need enforcement in-process, synchronous, checked before each API call — not after the response returns.

SDK-level budget enforcement wraps the model client and raises an exception synchronously before the next API call when cumulative spend crosses its limit. This is structurally different from circuit breakers (which evaluate after calls complete) and from monitoring (which observes the process from outside it entirely).

Two open-source libraries implement the pattern. agentbudget^[8] patches the Anthropic and OpenAI SDKs, tracks every call in dollar terms, fires soft-limit callbacks for warning, and raises BudgetExhausted before the next request when the hard limit hits. It also detects loops — repeated-call patterns within a configurable time window — and trips the breaker before cost can accumulate. tokencap^[10] wraps the client and tracks in token counts rather than dollars. Token counts stay accurate when providers reprice. Dollar equivalents go stale.

The OpenAI Agents SDK exposes usage metadata on every run — input tokens, output tokens, cumulative session totals — via RunHooks and the context object passed to each hook.^[14] That gives you the instrumentation data. It does not give you enforcement. You implement the pre-call check yourself, or use agentbudget/tokencap to wrap it.

Scope matters as much as mechanism. A global token limit that trips when any session exceeds budget denies service to every concurrent session the moment one runaway hits its ceiling. Per-session enforcement — one budget instance per agent run, never shared — means the 200 clean sessions running alongside one runaway never notice. The broken session trips its own limit. Everyone else keeps working.

Set the limit in tokens, not dollars. Token counts come from provider response metadata and stay correct. Dollar limits derived from per-token prices silently degrade when providers change pricing — and providers change pricing often. Translate your session budget to tokens once at configuration time, then enforce in tokens.

The 63-incident catalog^[11] documents a failure class, not a string of bad luck. Every incident had monitoring. None had synchronous enforcement. The teams were not negligent — they were using the standard observability stack, which is correctly designed for stateless services. Agent sessions are not stateless. They accumulate context, they retry, they hand off. The cost model has to match the execution model.

Fix the model first. Build the distribution in staging, set the 3× P95 limit, wire a per-session budget instance before you deploy. Then keep your monitoring — not as a cost control, but as the post-incident audit trail it is actually good at. The loop that already ran is expensive. The one you prevent is free.