Average Cost Is a Lie. The Tail Is Where the Money Goes.

Two LangChain agents burned $47,000 over eleven days in November 2025.^[1] One generated queries. The other validated responses. They locked into a handshake. Neither was malfunctioning — they did exactly what their prompts instructed. The validator kept rejecting the generator's output as incomplete. The generator kept trying again. Eleven days. The burn rate became visible when someone finally opened the billing dashboard.

Monitoring was in place. It did not stop the loop. Cloud cost anomaly detection aggregates spend over 24-48 hour windows — by the time a threshold fires, a session burning $0.045 per minute has been running for days.^[2] Monitoring that catches a runaway 36 hours in does not prevent $47,000 in damage. It documents it.

The real failure happened before any code ran. The team modeled expected agent cost as an average: typical tokens times model price equals expected session cost. That formula is correct for the 65-70% of runs that complete cleanly. It has no term for the 2-5% of runs where termination conditions fail and context grows without bound. In multi-agent systems, that small tail is where almost all your money goes.

More monitoring will not fix this. Tighter alerts will not fix this. The fix is upstream: model cost as a distribution, not a point estimate, and put enforcement synchronously in-process — checked before each API call — instead of asynchronously from outside it.

Standard API cost modeling treats requests as deterministic: hit the backend, return a response, pay a predictable amount. Agentic workflows do not behave that way. Tool calls per session are variable. Retry cycles are variable. Context length — which sets the cost of every subsequent call — grows on every iteration.

An ICLR 2026 study of agentic coding found that for identical task specifications, some runs consumed 10 times more tokens than others. There was no quality bonus. The expensive runs were less accurate on average.^[3] Same task, same agent, different inputs, 10x spread in cost. No mean compresses that into a useful budget.

Briefcase AI analyzed 1.4 million production LLM conversations. P95 cost ran 3-4x the median. The tail of massive conversations — about 9% of sessions — accounted for more than half of total spend.^[4] Model from the mean and you have silently excluded the expensive half of your bill.

Three tiers describe where agent cost actually lands. Happy path is the cheap, common case: the task completes in the tool calls and turns you predicted. Iterative search is the middle: retries, multi-hops, refinement on partial results. Edge-case recovery is the tail: a termination condition fails, context explodes, or two agents lock into the handshake that generates five-figure bills. Budget from your P95. Enforce at 3x P95. The edge-case tier is what the third multiple is there to catch — and it is exactly what every average-based budget pretends does not exist.

Multi-agent systems carry a cost property single-agent systems do not: context multiplies at every handoff. When an orchestrator passes work to a research agent, then forwards the result to an analysis agent, the analysis agent does not receive the research output alone. It receives the accumulated context of the entire session — the original request, the orchestrator's planning steps, the research agent's full output — as its input.

A 3-agent sequential pipeline where each agent produces 500-token outputs does not consume 3 x (1,000 input + 500 output) = 4,500 tokens. It consumes roughly 1,000 + 1,500 + 2,000 = 4,500 input tokens across the three agents, before any output tokens at all — a 4.5x multiplier on inputs alone, before retry overhead enters the picture.^[5] Group-chat patterns are worse. Five agents, ten rounds, 300 tokens per message: 15,000 tokens in shared context before any work happens, because every agent reads the full message history on every turn.

Chatbot cost intuitions do not transfer. Anthropic's own research finds agents burn 4x more tokens than direct chat, and multi-agent systems burn 15x more.^[6] One team migrated a simple RAG chatbot to an agentic pipeline. Monthly inference spend jumped from $4,200 to $31,000 — same underlying tasks, different architecture.^[7]

The handoff format is a cost decision. Full conversation history is the most expensive option. A compressed structured summary — key facts, decisions made, constraints that must persist — is cheaper and produces better downstream reasoning, because the receiving agent reads signal instead of noise. This is an interface design problem. Infrastructure cannot fix what the interface gets wrong.

Every team that hits a cost spiral reaches for the same fix first: better monitoring. The reflex is correct for visibility — you need to know what happened. It cannot solve prevention, because monitoring is structurally asynchronous.

Cloud cost alerts aggregate spend over 24-48 hour rolling windows. A session burning $10 per hour accumulates $240 before a daily anomaly threshold even has the data to fire on. That assumes a tightly-calibrated threshold. A team running legitimate large batch jobs alongside agents will eat persistent false positives, train its engineers to dismiss cost alerts as noise, and then miss the real event when it arrives. One engineer who traced this failure mode in a production multi-agent framework put it cleanly: 'Per-session accounting without a synchronous enforcement point tends to lag behind the actual spike. By the time you observe the overage, the burst has already happened.'^[9]

Prometheus-based dashboards have the same property. Even at one-minute scrape intervals, an alert rule must evaluate, match a condition, and route a notification before anyone can act. A zombie agent stuck in a reasoning loop burns $4-5 in a single query.^[7] Multiply by concurrent sessions and the 30-90 minutes it takes for an alert to reach someone with permission to kill the process. Monitoring is necessary. It is not sufficient.

The dashboard sees outputs. It cannot stop the process in flight. For that you need enforcement in-process, synchronous, checked before each API call — not after the response returns.

SDK-level budget enforcement wraps the model client and raises an exception synchronously before the next API call when cumulative spend crosses its limit. This is structurally different from circuit breakers (which evaluate after calls complete) and from monitoring (which observes the process from outside it entirely).

Two open-source libraries implement the pattern. agentbudget^[8] patches the Anthropic and OpenAI SDKs, tracks every call in dollar terms, fires soft-limit callbacks for warning, and raises BudgetExhausted before the next request when the hard limit hits. It also detects loops — repeated-call patterns within a configurable time window — and trips the breaker before cost can accumulate. tokencap^[10] wraps the client and tracks in token counts rather than dollars. Token counts stay accurate when providers reprice. Dollar equivalents go stale.

Scope matters as much as mechanism. A global token limit that trips when any session exceeds budget denies service to every concurrent session the moment one runaway hits its ceiling. Per-session enforcement — one budget instance per agent run, never shared — means the 200 clean sessions running alongside one runaway never notice. The broken session trips its own limit. Everyone else keeps working.

Set the limit in tokens, not dollars. Token counts come from provider response metadata and stay correct. Dollar limits derived from per-token prices silently degrade when providers change pricing — and providers change pricing often. Translate your session budget to tokens once at configuration time, then enforce in tokens.