The Agent Observability Framework Nobody Ships

A compliance-checking agent ran for 18 days before anyone noticed. HTTP 200 on every request. Valid JSON on every response. No alert fired. The first signal was a regulatory audit finding: the agent had been citing fabricated policy references since a retrieval index update three weeks earlier.

The instrumentation layer was complete by standard measures — latency tracked, error rates monitored, token costs attributed. What it never captured: whether responses were grounded in retrieved documents, whether a fallback path had been silently invoked, or whether downstream systems could actually use the output. Eighty-eight percent of production agent failures trace to infrastructure gaps — absent monitoring, missing guardrails, inadequate trace instrumentation — not model quality.^[1] The model was working exactly as designed. The diagnostic layer was never built.

Behavioral telemetry is the instrumentation layer between infrastructure metrics and output quality. It captures execution-time signals — grounded, fallback, confidence, downstream appropriateness — that describe how an agent behaved on a specific run, not just whether it returned 200. Without it, detection tells you something is degrading. Diagnosis remains impossible.

This is the four-step pipeline that teams research, partially prototype, and rarely complete: trace collection with behavioral signals → failure clustering → root cause attribution → eval generation. Each step compounds the last. The fourth step is the one that actually prevents recurrence.

Standard OpenTelemetry GenAI semantic conventions cover four things for every LLM call: model, token counts, latency, and finish reason.^[8] These are infrastructure signals. They tell you the bytes arrived on time and under budget. They have no opinion on whether the agent's reasoning was grounded in retrieved evidence, whether it silently fell back to a degraded path, or whether the output was structured in a way downstream systems could consume.

Behavioral telemetry fills that gap. It is the set of model-free, execution-time signals that describe how the agent behaved — not just that it completed. Each signal attaches as a span attribute on the existing invoke_agent span. No new infrastructure. No added latency. Richer attributes on spans you are already emitting.

Four signals cover the most frequent classes of silent failure:

agent.behavioral.grounded — Did the agent's response draw from retrieved context, or from its training weights? For any agent that does retrieval, this is the single most important behavioral signal. A grounded rate that drops from 0.94 to 0.71 over two weeks is a retrieval degradation signal that no infrastructure metric will surface. Compute it by checking whether retrieved document IDs appear in the reasoning trace or output attribution — a set membership check, not a model call.

agent.behavioral.fallback — Did the agent invoke a fallback path (secondary tool, default response, escalation trigger) rather than completing the primary workflow? A rising fallback rate says the primary path is failing silently. The agent is handling the situation, but not as designed. This catches silent workflow degradation before output quality metrics do.

agent.behavioral.confidence — What was the agent's assessed certainty about its tool selection or decision? Some providers return logprobs for tool selection; others require a confidence estimation prompt. This signal catches the plausible-but-wrong failure class — outputs the agent produced while uncertain, which are the first candidates for downstream validation or human review.

agent.behavioral.downstream_ok — Could a downstream system actually use this output? Schema validation passes but field values are malformed. A date is in the wrong timezone. A required ID is null. This catches outputs that are structurally valid but operationally useless — the failure class that hallucination detection misses because the format is correct and the content is plausibly wrong.

Detection gives you a list of failed runs. That list, unprocessed, is not actionable. A team investigating 40 individual failures will converge on nothing useful. A team investigating three failure clusters with 13 members each will converge on root causes in under an hour.^[4]

Failure clustering groups failed runs by behavioral signature — shared failure type, common signal pattern, same execution step, similar tool call sequence — to surface the underlying issue rather than individual incidents. The operational test: when you look at the cluster and immediately recognize the common cause, the clustering is working. When each member looks different, the signal dimensions need refinement.

The bootstrapping challenge is real. You cannot build a failure taxonomy before you have seen failures, and you cannot cluster failures without a taxonomy. The approach that works:

Open coding first. Have a domain expert read 20–50 recent failed traces without a category framework. Write unstructured observations. Do not try to categorize — just observe. The question at this stage is: "What went wrong here?" not "Which category does this belong to?"

Axial coding second. Group open-coded observations into a tentative taxonomy. Count occurrences per cluster. Re-read the largest clusters and confirm the grouping is coherent. Repeat until new traces stop producing new categories — that is the point at which you have enough taxonomy coverage to automate.

Research on production agent fault characterization across 13,602 documented failures identified five behavioral signal patterns that cluster reliably: grounding failures, fallback rate spikes, confidence collapses, downstream rejection events, and tool call anomalies.^[7] Each maps to a distinct root cause class. The taxonomy is the key artifact. Building it before you have enough failures means it will miss the most common modes. Automating clustering before you have a validated taxonomy generates better-organized noise.

For teams early in the process: read the first 20 failures manually. At 20 failures per week, manual review is still tractable and produces a better taxonomy than premature automation. The automation becomes worth the investment when the weekly volume exceeds what a human can review — around 50–100 failures per week depending on team bandwidth.

Root cause attribution is where behavioral signals earn their instrumentation cost. Without them, you have a failure timestamp and an output. With them, you have a behavioral fingerprint of the run: what grounding rate the agent maintained, whether it invoked fallback, where confidence dropped, and whether the downstream output passed validation.

The most common diagnostic error is investigating the failure node rather than tracing backward to its cause. Research on multi-step agent workflows found that 63% of step-level failures are propagated from upstream errors — not locally caused.^[3] When a downstream synthesis step produces a hallucinated fact, the root cause is often a retrieval failure or a tool argument malformation three steps earlier. Debugging the synthesis step finds nothing. Following the behavioral signal trail backward finds the break. Root cause identification improved from a median of 4.2 hours to 22 minutes with structured diagnostic frameworks.^[3]

Four root cause patterns account for the majority of silent production failures:^[4]

Provider model drift. Confidence drops and the execution fingerprint shifts — more fallback invocations, different tool selection ratios — but no code or prompt changed on your side. Correlate with the provider's changelog. A confidence collapse on a day you deployed nothing is a provider-side model update until proven otherwise.

Prompt regression. Grounded rate drops and downstream_ok falls. Correlate with prompt hash changes. A silent template change that removed a constraint produces exactly this: the model still runs, the tools still call, but responses stop being grounded and stop satisfying downstream validators.

Retrieval drift. Grounded rate drops while confidence stays stable. The agent attempts retrieval, finds nothing useful, and proceeds on weights. Check the retrieval index for changes: new documents that diluted relevance, stale embeddings, a query expansion change that altered what gets retrieved.

Tool schema change. Fallback rate spikes and downstream_ok fails. An upstream API changed its response schema. The tool call succeeded. The output is parsed without error. The values are malformed or missing. This is the failure mode that schema validation at the tool boundary would have caught and almost never does.

The eval generation step is what most teams skip. They diagnose the root cause, fix the immediate problem, and close the incident. Three months later, after a model upgrade or a prompt refactoring, the same failure ships again. Nobody connects it to the previous incident because the previous incident produced no artifact.

The pattern that prevents recurrence: every diagnosed root cause produces one eval case. Every eval case gets a severity and a grader.^[5]

Severity determines CI behavior: P0 blocks the deploy. P1 warns in CI and requires an explicit override. P2 logs and tracks. Safety violations and compliance failures are P0 regardless of frequency. Quality regressions and grounding failures are P1. Formatting issues and downstream validation failures are P2 until they affect a compliance workflow.

Grader type follows failure class. Deterministic failures — the agent cited a document ID that was not in the retrieved set, the output JSON failed schema validation, fallback was invoked on a task type that should never trigger fallback — get assertion-based graders. Cheap to run, cheap to maintain, exact on recurrence. Semantic failures — the agent produced a plausible but factually wrong synthesis — get LLM-as-judge graders. More expensive, needed for the cases where exact matching will never work.

The compounding effect is real. Chronicle Labs documented a team that started with 23 manually written evals and grew to 147 automatically generated from production failures over 60 days.^[5] Regression coverage grows as a function of production failure rate — not developer time. The more failures the system sees, the harder it becomes to regress on known failure modes.

One honest constraint: evals generated from production failure clusters cover failure modes the agent has already exhibited. They do not catch novel failure modes. The production-to-eval loop needs companion coverage from adversarial testing and boundary analysis for failure classes the agent has not yet seen but will eventually encounter.

Signal-based trajectory sampling to select which failures warrant eval cases improves the efficiency of this work. Research on agent trajectory triage found that signal-based sampling achieves 82% informativeness versus 54% for random sampling — a 1.52× efficiency gain per informative trajectory selected.^[2] The behavioral signals you already collected at execution time are the selection criteria. Grounding failures with downstream_ok=false and confidence below 0.70 are the highest-priority candidates. Clean runs with all signals in normal range contribute nothing new to the eval corpus.

The framework is not architecturally complex. Four behavioral attributes on existing spans. Weekly failure cluster reviews. A root cause lookup table that fits in a spreadsheet. One eval case per diagnosed incident, wired into CI.

What makes it hard is the organizational habit of closing incidents at the fix stage. The eval generation step looks like overhead until the same failure ships again after a model upgrade six weeks later — and nobody connects it to the previous incident because the previous incident produced no artifact.

The test of whether your observability stack is complete is not whether you can detect degradation. Detection is the minimum. The test is whether every production failure that passes through the diagnostic pipeline becomes a constraint on the next deploy. If it does not, the loop is not closed. The failure is waiting for its next deployment.