The Token Cost Trap: How Multi-Agent Systems Spiral Into Overspend

In November 2025, a LangChain research pipeline burned $47,000 over eleven days.^[1] Two agents — one generating queries, the other validating responses — locked into a handshake loop that never terminated. Neither agent was malfunctioning. Both did exactly what their prompts instructed. The validator kept finding the generator's output incomplete. The generator kept trying to improve it. Back and forth, for 11 days, at a burn rate that only became visible when someone finally checked the billing dashboard.

The team had monitoring in place. It did not stop the loop. Cloud cost anomaly detection aggregates spend over 24–48 hour windows — by the time a threshold fires, a session burning $0.045 per minute has already been running for days.^[2] Monitoring that fires 36 hours into a runaway doesn't prevent $47,000 in damage. It documents it.

The real failure happened before any code ran. The team modeled their expected agent cost as an average: typical tokens times model price equals expected session cost. That formula is correct for the 65–70% of runs that complete cleanly. It has no term for the 2–5% of runs where termination conditions fail and context grows without bound. In multi-agent systems, that small tail accounts for nearly all your money.

This isn't about adding monitoring or tightening alerts. It's about the upstream problem: cost-by-distribution thinking instead of cost-by-average thinking, and enforcement that operates synchronously inside your process rather than asynchronously from outside it.

Standard API cost modeling treats requests as having deterministic cost profiles: a request hits a backend, returns a response, costs a predictable amount. Agentic workflows don't work that way. The number of tool calls per session is variable. Retry cycles are variable. Context length — which determines the cost of every subsequent call — grows with each iteration.

An ICLR 2026 paper analyzed token consumption across agentic coding tasks and found that for identical task specifications, some runs consumed 10 times more tokens than others, with no corresponding quality benefit. Higher token usage was actually associated with lower task accuracy on average.^[3] The same task, the same agent, different inputs — and a 10x spread in cost. You cannot average that distribution into a useful budget.

Briefcase AI's analysis of 1.4 million production LLM conversations found that 95th percentile costs exceed the median by a factor of 3–4x, and that the tail of massive conversations — roughly 9% of total sessions — accounts for over half of total cost.^[4] Model from the mean and you've invisibly excluded the expensive half.

Three tiers describe where agent costs actually land. The happy path is your cheapest, most common case — task completes in expected tool calls and turns. Iterative search is the middle tier: the agent retries, multi-hops, or refines based on partial results. Edge-case recovery is the tail: a termination condition fails, context explodes, or two agents enter the handshake pattern that generates five-figure bills. Budget from your P95. Enforce at 3× P95. The edge-case recovery tier is what the third multiple is there to catch — and it's also what every average-based budget ignores.

Multi-agent systems have a cost property that single-agent systems don't: context multiplies at every handoff. When an orchestrator passes work to a research agent, then forwards the result to an analysis agent, the analysis agent doesn't receive only the research output. It receives the accumulated context of everything that has happened in the session — the original request, the orchestrator's planning steps, the research agent's full output — as its input.

A 3-agent sequential pipeline where each agent produces 500-token outputs doesn't consume 3 × (1,000 input + 500 output) = 4,500 tokens. It consumes roughly 1,000 + 1,500 + 2,000 = 4,500 input tokens across the three agents, before any output tokens — a 4.5× multiplier on inputs alone, even before accounting for any retry overhead.^[5] Group-chat patterns are worse: a 5-agent group chat running 10 rounds at 300 tokens per message burns 15,000 tokens in shared context alone before any actual work occurs, because every agent reads the full message history on every turn.

This is why chatbot cost intuitions don't transfer. Anthropic's own research found that agents typically burn 4 times more tokens than direct chat interactions, and multi-agent systems burn 15 times more.^[6] One team that migrated from a simple RAG chatbot to an agentic pipeline watched their monthly inference spend jump from $4,200 to $31,000 — same underlying tasks, different architecture.^[7]

What you pass at each agent boundary is a cost decision. Full conversation history is the most expensive handoff format. A compressed structured summary — key facts, decisions made, constraints that must persist — is cheaper and often produces better downstream reasoning, because the receiving agent gets signal rather than noise. This is an interface design problem, not just an infrastructure one.

Every team that hits a cost spiral reaches for the same immediate fix: better monitoring. That reflex is correct for cost visibility — you need to understand what happened. It cannot solve the prevention problem, because monitoring is structurally asynchronous.

Cloud cost alerts aggregate spend over 24–48 hour rolling windows. A session burning $10 per hour accumulates $240 before a daily anomaly threshold can fire. And that assumes your threshold is calibrated tightly — a team that also runs legitimate large batch jobs will face persistent false positives, train engineers to dismiss cost alerts as noise, and then miss the real event. One engineer who traced a similar failure in a production multi-agent framework put it directly: 'Per-session accounting without a synchronous enforcement point tends to lag behind the actual spike. By the time you observe the overage, the burst has already happened.'^[9]

Prometheus-based cost dashboards face the same constraint. Even at one-minute scrape intervals, an alert rule must evaluate, match a condition, and notify before any action is possible. A zombie agent stuck in a reasoning loop can consume $4–5 in a single query.^[7] Multiply that by concurrent sessions and by the 30–90 minutes it takes for an alert to reach someone who can act, and you understand why monitoring is necessary but not sufficient.

The monitoring sees outputs. It cannot stop the process mid-flight. For that, you need enforcement that operates in-process, synchronously, checked before each API call — not after.

SDK-level budget enforcement wraps the model client and raises an exception synchronously before the next API call is made, when cumulative token spend crosses its limit. This is fundamentally different from circuit breakers (which check session cost and trip after calls complete) and from monitoring (which observes cost from outside the process entirely).

Two open-source libraries implement this pattern. agentbudget^[8] patches the Anthropic and OpenAI SDKs to track every call in dollar terms, with soft limits that fire warning callbacks and a hard limit that raises BudgetExhausted before the next request is made. It also adds loop detection — configurable repeated-call patterns within a time window that trip the breaker before cost can accumulate. tokencap^[10] wraps the client and tracks in token counts rather than dollars, which is more stable across provider pricing changes since token counts are always accurate while dollar equivalents become stale after pricing updates.

The enforcement scope matters as much as the mechanism. A global token limit that trips when any session exceeds budget will deny service to all concurrent sessions when one runaway agent hits its ceiling. Per-session enforcement — one budget instance per agent run, never shared — means the 200 clean sessions running alongside one runaway are unaffected. One broken session trips its own limit; everyone else keeps working.

Set your limit in tokens, not dollars. Token counts come directly from provider response metadata and never become stale. Dollar limits calculated from per-token prices silently degrade after pricing changes — and providers change pricing frequently. Translate your session budget to tokens once at configuration time, then enforce in tokens.