Prompt Contract Versioning: Stop Breaking Multi-Agent Systems on Deploy

The most dangerous prompt change in a production multi-agent system is the one labeled "minor cleanup." Not a model swap. Not a new tool integration. A platform team added a sentence to a customer support agent's system prompt to clarify escalation tone. They ran five manual tests. It looked better. They shipped it.

Three days later, a routing orchestrator upstream began misclassifying 12% of escalated tickets — because that orchestrator was extracting a priority signal from a specific clause the "cleanup" had reworded. Nobody had documented the dependency. It surfaced in ticket quality data two business cycles later, not in monitoring.

That failure pattern has a name: an implicit output format dependency in a downstream consumer, broken by an undocumented prompt change. It's the core problem that prompt contract versioning exists to solve.

Prompt contract versioning treats every production system prompt as a versioned interface with defined downstream consumers, documented output contracts, and explicit breaking change classification. The discipline borrows directly from API versioning. The mechanics need adaptation for probabilistic, prose-heavy LLM outputs — but the governance logic is identical.

Prompt registries — the standard tooling recommendation for this problem — manage versions and enable rollback. What they don't do: tell you which downstream systems depend on specific output fields, or block a change that removes those fields before it reaches production. That gap is where silent failures live.

Software engineering has a well-developed vocabulary for interface contracts. When a REST API removes a field, the provider bumps a major version, communicates the deprecation, and gives consumers a migration window. The contract between provider and consumer is explicit, versioned, and observable.

Production AI agents have a contract too. Almost nobody treats it like one.

An agent that produces structured output — a JSON response, a formatted report, a decision with specific field names — is an interface. Anything consuming that output has a dependency on its format and behavior. The dependency is real whether it's documented or not. Teams discover this the hard way when a routing orchestrator breaks because the research agent it calls changed its citation structure, or a summarization pipeline starts injecting wrong data because an upstream classifier's output schema quietly shifted.

The gap compounds in multi-agent systems. A routing orchestrator is written to parse what the upstream classifier "happens to produce." A summarization pipeline is built against whatever the research agent currently outputs. These implicit dependencies accumulate as teams iterate on prompts independently — often without knowing who else is consuming the output. The result: a web of undocumented interface assumptions that nobody owns and nobody tests.

The failure mode has two layers. First, structural drift: the output schema changes enough that downstream parsing code throws an exception or extracts None where it expected a string. This surfaces quickly. Second, semantic drift: the schema is valid, values are present, but their meaning has shifted in ways the consuming agent wasn't designed to handle — a confidence score that used to range 0.0–1.0 now sometimes returns null for uncertain cases, or a tone classification that previously returned neutral now returns formal/neutral to be more precise. No exception fires. Quality degrades invisibly.

Prompt registries tell you what version is running and let you roll back. They don't enforce contracts between producers and consumers. A registry tells you which version is active; a contract system tells you which downstream consumers will break if you change it — and blocks the change in CI if any will. You need both. ^[2]

Semantic versioning adapted for system prompts gives teams a shared vocabulary for what counts as a breaking change. Without it, every prompt change is a judgment call. Judgment calls at scale become inconsistent — some breaking changes ship unannounced, some patch changes get escalated unnecessarily.

The counter-intuitive rule: a "typo fix" is a patch only if no downstream consumer depends on the exact phrasing you fixed. If a downstream parser was extracting a value by matching on a specific string that happened to contain the typo, fixing it is a major version bump — regardless of your intent. Classification is determined by the consumer's dependency, not the author's perception of the change. ^[3]

This matters because most prompt authors underestimate how many downstream systems parse around specific output phrasing. The practical approach: before classifying any change as patch, ask whether any registered consumer test touches the modified clause. If you don't have consumer contracts registered yet, ask who consumes this agent's output and contact them before shipping.

Constrained decoding — now supported natively by Anthropic (public beta, November 2025), OpenAI (August 2024), and Google (late 2024) — shifts structural contract enforcement to the generation layer by compiling JSON Schema into a finite state machine applied during token sampling. The model literally can't produce tokens that violate your schema. ^[6] This eliminates one class of structural breaking changes (malformed output) but leaves semantic drift fully unaddressed. Don't mistake schema enforcement for contract enforcement.

The failures that matter are two or three hops deep.

A common structure: an orchestrating agent calls a research agent, extracts structured data from the response, and passes it to a summarization agent. The research agent's team updates their system prompt — "minor clarification of citation format." The orchestrating agent was parsing a priority field that reliably appeared in position three of the citation block. After the update, the citation structure shifted enough that position three now contains a different value. The orchestrator injects wrong data into the summarization agent's context. The summarization agent produces plausible but incorrect summaries. No exception is thrown. No alert fires. The divergence surfaces in a quality audit three weeks later.

Three properties make this failure class dangerous: it is silent (no exceptions, no degraded health indicators), it is delayed (surfaced through output quality rather than system metrics), and it is deep (three hops from the change to the visible symptom). Standard observability — error rates, latency, uptime — won't catch this. Only a registered consumer contract, running as an automated check on every prompt change, would have blocked the update before it shipped.

This isn't hypothetical. Research on LLM agent failure attribution shows that small logic errors in upstream agent outputs can propagate across multi-hop workflows, with the compounding effect making root cause identification significantly harder than in single-agent systems. ^[7] A practical rule that has emerged from teams who've been through this: if Agent B's behavior depends on Agent A's output format in any way, changing Agent A's prompt is a major version bump — regardless of how the change feels to Agent A's team.

The agent manifest is a document that maps each production agent to the contracts it publishes and the contracts it depends on. Its purpose isn't documentation — it's searchability. When a prompt change is proposed, the manifest answers immediately: which downstream systems depend on this agent's output format? Without a manifest, answering that question requires a codebase audit, trace log parsing, and conversations with teams who may not remember what they built. ^[3]

The manifest has two halves. The provider half records which version of the prompt is active, what output schema and fields the current version produces, and when each field was introduced. The consumer half records which upstream agents' outputs this agent depends on, which fields it extracts, and which version of the upstream contract it was last tested against.

Consumer contracts — the test fixtures that block breaking changes in CI — are registered within the manifest. Each registered consumer specifies: given this input, the upstream agent's output must contain these fields with these types. The platform team runs these fixtures against every candidate prompt version before promotion. One failing fixture blocks the deployment.

This pattern is well-established for REST APIs under the Consumer-Driven Contract (CDC) model, popularized by tools like Pact. Applying it to prompts requires adapting to probabilistic outputs — you test structural contracts (field presence, type, schema conformance), not exact value matches. ^[1] Contract testing for AI pipelines borrows these patterns from microservices and focuses on structural guarantees rather than value equality — the structural orientation leaves semantic drift unaddressed by design, which is why shadow validation is a required companion step. ^[4]

The most common mistake teams make when first implementing CDC for prompts: writing tests that assert on content values rather than output structure. "The response must contain the phrase 'high priority'" is a fragile assertion that will fail on any benign prompt improvement. It's also useless as a contract — it prevents improvement without protecting against real breaks.

A contract test for LLM output makes structural assertions: fields are present, types are correct, required values are non-null, arrays have the expected element shape. These assertions are stable across prompt versions because they test the output schema, not the output value. They'll catch a real breaking change — a renamed field, a changed type, a removed required property — while ignoring benign content variation.

Here's a concrete Python fixture using pytest and the Anthropic client. The consuming team owns this file; the producing team's CI runs it:

The market for prompt management tooling has matured rapidly, but the product categories serve different jobs. Conflating them is how teams end up with a false sense of coverage.

MLflow Prompt Registry (3.0+) stores versioned prompt templates with Jinja2 variable support, environment aliases (production, staging), and CI integration via mlflow.genai.load_prompt("prompts:/customer-research@production"). Aliases let you promote versions without redeploying application code. It doesn't know which downstream systems depend on which prompt versions. ^[5]

LangSmith Prompt Hub uses commit-hash-based versioning — pin to a specific hash (my-prompt:abc1234) for reproducible production deployments. Webhooks can trigger CI pipelines when prompts are updated. It also doesn't model consumer dependencies or run consumer contract tests. ^[8]

Braintrust, PromptLayer, Maxim AI extend version management with eval integration, A/B testing, and rollback. Still no consumer dependency graph.

None of these tools implement the consumer dependency graph. That layer — the agent manifest plus CDC test fixtures — is currently implemented by teams themselves, typically as YAML manifests in a shared infrastructure repo with contract tests in the producing agent's CI pipeline.

The practical implication: you need both a registry (for storage, rollback, and alias-based promotion) and a CDC layer (for consumer protection). They're complementary, not redundant.

The contract test fixtures are useless unless they run automatically on every candidate prompt version. Here's a minimal GitHub Actions job that pulls a candidate prompt, sets it as the CANDIDATE_PROMPT_PATH environment variable, and runs all registered consumer contracts. The job fails — and blocks promotion — if any contract assertion fails.

Not every agent needs the full CDC layer. The overhead of maintaining manifests and contract tests is real — and it's only worth paying when the dependency graph justifies it.

Apply the full discipline — immutable versions, manifest, registered CDC tests, shadow validation — when:

• Two or more teams share an agent's output (organizational boundary means communication is async)
• An agent's output is consumed by automated systems rather than humans
• An agent's output drives decisions with downstream consequences (routing, escalation, pricing)
• The agent has been in production for more than 90 days and its consumers have diverged from the original design

A lighter version — immutable versions + explicit classification + consumer review on major bumps — is sufficient when:

• A single team owns both the producing and all consuming agents
• All consumers are human (no automated parsing of output structure)
• The agent is experimental or in active redesign (spending on contract infrastructure for a prompt you'll rewrite next week is waste)

The threshold to cross from light to full is simpler than it sounds: the moment you can't answer "who parses this agent's output and what do they extract?" from memory in 30 seconds, you need the manifest.

The pattern described here — manifests, CDC tests, shadow validation — isn't novel engineering. It's standard microservices discipline applied to a new substrate. The novelty is that most teams building multi-agent systems don't think of prompts as interfaces, so they don't apply interface discipline to prompt changes. The result is a production environment where the most consequential code changes — the ones that alter agent behavior across an entire pipeline — go through less governance than a CSS tweak.

That's the actual problem this discipline solves. Not technical complexity. An assumption that prompts are configuration, not contracts — and that the distinction doesn't matter until a routing orchestrator starts misclassifying 12% of tickets because someone improved a tone instruction.