Context Store for AI Agents: The Governed Data Layer Agents Actually Need

A support routing agent classifies enterprise customers using a column that was semantically reclassified eighteen months ago. The values still look correct. The column still exists. The rule changed. Nobody updated the record because nobody owned the record. Enterprise accounts get routed to the wrong queue for weeks before someone notices the pattern in escalations.

The model did not malfunction. It executed against the data it was given, applying the only meaning it could infer from the schema. The constraint that would have caught the error — use customers.currenttier, not orders.customertier for routing — exists only in the heads of the four people on the data team. There is no formal representation an agent can query. So the question routes back to a human, or it doesn't get asked at all.

This is a data sovereignty problem in the precise sense: the rules that make data mean something are concentrated in a small number of people and have no executable shape. Every agent question that touches business semantics turns into a Slack thread. The data engineer becomes the bottleneck. Adding more agents multiplies the load proportionally.

A context store is the structural answer — a versioned, governed, freshness-monitored layer that encodes what data means and what constraints apply, so agents query it directly instead of routing questions back to humans. Data engineers ship semantic schemas the way they ship dbt models. This article covers what goes in one, how to structure the schema, how to keep it current, and the ownership inversion that turns the data team from per-question answerers into per-schema publishers.

Research on NL2SQL agents matches what every practitioner who has shipped one already knows: data agents fail not from retrieval errors or model limitations, but because they form misconceptions about what data actually represents.^[1] They query the right column for the wrong concept. They join on a deprecated ID field. They ignore the rule that inventory counts below 5 are unreliable due to sync lag from a legacy ERP. The model did nothing wrong. Nobody encoded the rules.

The performance gap between benchmark and production is stark. Models reach 75–85% execution accuracy on clean academic benchmarks like Spider and BIRD — but in real enterprise environments with unfamiliar schemas and domain-specific logic, that number collapses. The SIGKDD NL2SQL-BUGs study found that roughly 25% of failures on curated benchmarks trace to semantic errors alone — in real databases with more complex schemas and diverse query patterns, the rate is higher.^[10] Tk-Boost, a tribal knowledge injection framework, recovers up to 16.9% accuracy on Spider 2.0 and 13.7% on BIRD by pre-loading agents with the institutional rules that production databases carry but never document.^[1]

Meta hit this at scale. Across 4,100+ code files in three repositories, only 5% had any form of codified tribal knowledge — the rules experienced engineers applied automatically but had never written down. To close the gap, they built a pre-compute engine using 50+ specialized AI agents that systematically read code files and produced context documents encoding what the data meant, what to watch for, and how to navigate edge cases.^[2] Coverage went from 5% to 100% of code modules.

Most teams can't run 50 agents to bootstrap. They face the same structural problem regardless. A data engineer who already spends 30% of the week answering semantic questions from analysts won't survive a system where agents ask the same questions ten times faster. The Datadog State of AI Engineering report found that 69% of all LLM input tokens in production traces were consumed by system prompts — internal instructions and policy definitions pumped in from outside because the underlying data layer carries no semantics of its own.^[9] Teams are compensating for a missing context layer by stuffing it into prompts. The questions have to stop routing to humans. The only way to stop them is to publish the answers as queryable artifacts before the agent asks.

A context store is a versioned, governed semantic layer that gives AI agents the business rules, constraints, and freshness metadata they need to interpret data correctly — as a queryable, owned artifact, not as institutional folklore.

It is not a RAG knowledge base. RAG retrieves chunks of text by semantic similarity. A context store serves governed rules with explicit ownership, version history, and freshness SLAs. The operational difference is unforgiving: a RAG document goes stale silently and retrieval quality drifts. A context store document has an owner who is responsible for keeping it current, and a freshness state agents observe at query time. The two are complementary, not competitors. RAG handles unstructured retrieval. The context store handles governed business rule delivery for operational decisions.

It is not a data catalog either. Catalogs describe what data exists and where. A context store encodes what data means and what constraints apply — the rules that can't be derived from a column description. Catalogs are inventory. Context stores are doctrine.

The Atlan context engineering framework treats every piece of context as a versioned, auditable data product with the same lifecycle and ownership discipline you'd apply to a core dbt model.^[4] The a16z framing is blunter: agents need a business knowledge layer that answers the questions raw data can't.^[5]

Context Kubernetes, the research architecture for managing enterprise context at scale, names the components precisely: a Context Registry managing document identity, a Freshness Manager tracking four states (fresh, stale, expired, conflicted), a Permission Engine enforcing RBAC, and a Trust Policy determining which documents agents can act on without human verification.^[6] These are the infrastructure primitives. The discipline they imply is the actual product.

Most teams that try to build a context store start by porting their existing data documentation — column descriptions, table readmes, dictionary entries. The output is a searchable knowledge base. It's not a governed context store, because it's missing the three fields that make a document agent-safe: constraints, freshness, and owner.

The constraints field is the one almost nobody covers. It encodes what agents must not do with this data — the negative knowledge that lives in experienced engineers' heads and routinely causes failures when absent. A rule that says enterprise tier requires ARR >= $50K is useful. A constraint that says never use orders.customertier for current-state routing — use customers.currenttier is the difference between a routing agent that works and one that silently misclassifies for months.

The freshness object anchors the SLA. Without it, the context document is a snapshot with no expiry. Agents that can see freshness state include uncertainty in their output when a rule is stale — instead of making confident wrong decisions.

A minimal but complete schema for a context document:

Freshness is where most context store implementations rot in production. Teams build the initial schema, populate it with the rules that exist on day one, and ship. Six months later the business has changed, the documents haven't, and agents are confidently applying retired rules to live decisions. The failure mode is structural: degradation is silent by default. A stale document doesn't throw. It serves wrong information with full confidence.

The four-state model from the Context Kubernetes architecture cuts the problem cleanly:^[6]

• Fresh — verified against its source within the SLA window. Agents act on it.
• Stale — the SLA window passed but the document hasn't been explicitly invalidated. Agents flag uncertainty in their output.
• Expired — the owner must re-verify before the document can be used. Treat as unavailable.
• Conflicted — two documents make contradictory claims. A human resolves before any agent proceeds.

Different context types take different SLAs. Inventory and real-time pricing rules need freshness within minutes — a stale inventory rule causes agents to promise availability that doesn't exist. Financial reporting context tolerates daily freshness. Support routing rules sit in between and need near-real-time updates to avoid misrouting live cases.^[3] The SLA belongs in the schema itself, not in a separate monitoring config, because agents inspect it at query time to decide whether to use the document or escalate uncertainty.

The production-grade pattern is CDC-driven freshness, not polling. When dim_customers updates, the freshness monitor checks whether any context document references that model and transitions staleness state immediately.^[3] Polling hourly means you can be 59 minutes behind a breaking rule change. CDC-driven updates catch it in minutes. The cost difference shows up in a customer escalation.

The context store is useless if agents read raw YAML files. They need a queryable API that enforces access control, returns freshness state, and logs every lookup. The schema lives in Git; the API is the runtime surface.

A minimal FastAPI wrapper is enough to start. The important design decision is: freshness state must be in every response. Agents that receive a stale document need to know it at the time of the response, not after a human reads the audit log. Here's a pattern you can adapt:

The traditional data engineering model is reactive. Analysts ask questions. Engineers answer them. The throughput is bounded by how fast the engineer can context-switch and type a Slack reply. When agents enter the loop, that ceiling collapses. Agents generate questions continuously, with no natural throttle.

The ownership inversion changes the direction of the work. Data engineers become context publishers. They formalize the rules they currently hold in their heads into governed context documents — and agents query those documents directly. The engineer's output changes from verbal explanations to schema artifacts. The questions stop routing to humans because the answers are already published.

This is what a16z means by your data agents need context:^[5] data stops being a pipeline artifact and becomes a context product. One well-maintained context document serves a dozen agents consistently. The marginal cost of adding another agent drops near zero once the schema exists.

The friction is the quality of the initial formalization. Getting data engineers to write constraints[] fields — not just column descriptions — requires a different mental model: writing not just what the data is, but what can go wrong when an agent uses it incorrectly. That work is unfamiliar. Teams that rush it produce documents that are technically accurate and operationally incomplete. The implementations that work treat initial formalization as a sprint goal, not a backlog item, and pair engineers with agents during discovery to surface the questions agents actually ask before writing the schema.

The Atlan context engineering framework names the architectural answer to this drift: Phase 3 orchestration, which routes only the certified context product a query needs — blocking uncertified documents from reaching agents entirely.^[4] Once a document is published, it's the only version agents see. The certification step is the gate.

Teams building context stores often treat them as a general good-practice measure — which makes them hard to prioritize and harder to staff. The argument gets sharper when you name the specific failure modes they prevent.

Silent semantic drift. A business rule changes. The column doesn't. The agent keeps applying the old rule with full confidence. No exception is raised. The only observable signal is a downstream customer complaint or a skewed metric someone catches in a quarterly review. A freshness-monitored context document makes the drift visible: the source model updates, the document transitions to stale, the owning team gets an alert, and agents flag uncertainty on every decision until the document is re-verified.

Column aliasing. Two tables have columns named the same thing that mean different things in different contexts — customer_tier in orders is the tier at time of purchase; customer_tier in customers is the current tier. An agent with no context store queries either column depending on which it retrieves first. A context document with a constraints[] entry saying which column to use for which decision class eliminates the ambiguity entirely. This is the class of error the NL2SQL-BUGs benchmark was designed to measure — and it accounts for a material fraction of production failures on real enterprise schemas.^[10]

Threshold ossification. A business threshold (enterprise ARR cutoff, inventory reliability floor, latency SLA) was defined 18 months ago and encoded in 14 different places. Eleven of them got updated when the threshold changed. Three didn't. One of the three is a context store document that an agent queries. A versioned, owned document with a known source makes the update surface explicit. The owning team updates one artifact; every agent picks up the new version at next query.

Context stores are not new. Semantic layers and data catalogs have existed for years. What has changed is the ownership model and the cost of getting it wrong. When agents acted on queries one at a time with a human in the loop, a stale business rule caused one bad answer. When agents act continuously at scale, the same stale rule causes a systematic failure that compounds until someone reads the escalation log.

The teams that get this right are the ones where data engineers stop seeing themselves as answerers and start seeing themselves as publishers. The work is the same: encoding the rules they already know. The delivery is the difference. A Slack reply serves one analyst once. A schema artifact serves every agent that ever queries it — at near-zero marginal cost per query.

Start with the failures. Pick five agent decisions that went wrong in the last sprint and trace each one back to the missing or stale rule. Write those five as governed context documents with non-empty constraints[]. Set a freshness SLA. Ship them behind a simple REST API. Measure whether the same failures recur. That's one week of work for a clear signal about whether the pattern scales. The agents that stay shipped are the ones running on top of context the data team owns instead of remembers.