Agentic AI Safety: Five Enforcement Layers Between Your Agent and a Catastrophe

In July 2025, an engineer ran a routine change through an agent-assisted coding tool during an active code freeze. The agent deleted the production database. Not staging. Not a test instance. The live database paying customers were sitting on top of. The post-mortem called it a "catastrophic failure."^[1]

This was not a thought experiment from a safety paper. The incident landed in the AI Incident Database^[2], got covered by Fortune, and chewed through engineering Slack channels for weeks. It was not unique. The same month, an AWS Cost Explorer environment in a Mainland China region went offline for 13 hours — also traced to an AI coding agent that deleted and recreated a production environment. By late 2025, the pattern was visible from a hundred feet up: agentic systems shipped into production with permission scopes that wildly exceeded the work they were doing, and with nothing meaningful between intent and execution.

This is not a playbook for slowing adoption. It covers the five enforcement layers that stop a helpful automation from becoming an expensive incident review. Each layer maps to a specific failure mode that has already happened to someone. Build them now, or build them in the post-mortem.

The Replit database deletion^[1] is the loudest example, not the only one. The AI Incident Database (Incident 1152)^[2] catalogs the same shape across stacks and industries.

An agent with broad Terraform permissions runs a configuration change that includes a destroy on a production database. The IaC tool executes it faithfully — exactly what the agent asked for. The execution was not the failure. The permission scope that allowed a routine task to cascade into a destructive operation was.

A deployment agent rolls back a failed canary, picks a target three versions back, and reintroduces a security vulnerability that was patched two releases ago. Nobody had estimated the blast radius of the rollback path itself.^[5]

A cleanup agent reads "older than 90 days" one way, the team meant another, and 18 months of audit logs under legal hold get deleted. No dry-run step ever showed what the operation would touch.

These are not failures of model intelligence. They are failures of architecture. The agents executed competently inside the permissions they had. The gap between agent capability and consequence containment is where the incidents live.

The OWASP Top 10 for Agentic Applications — released in December 2025 with input from over 100 security experts and endorsed by NIST, Microsoft, and NVIDIA^[7] — formalizes what the incident record already showed: the two highest-risk agentic failure modes are Excessive Agency (an agent granted capabilities beyond the task it's doing) and Insufficient Authorization (no policy layer between the agent and a destructive tool call). Both are architecture failures, not model failures. Both map directly to layers 1 and 2 of this playbook.

Permission scoping means each agent runs with the minimum credentials its actual function requires. Nothing more. The reason this is the most-skipped layer is structural: nobody owns scope cleanup. One admin service account is faster to wire up than per-agent scoped credentials, and "we will tighten it later" never gets prioritized over the next feature.^[4]

The Replit incident happened in part because the agent held write access to production infrastructure during a code freeze.^[1] A scoped agent doing code review would have held read-only access on the codebase and zero infrastructure permissions. The deletion path simply would not have existed.

Scoping happens at three levels:

Tool-level — which tools the agent can invoke at all. A documentation agent does not get deployment tools.

Resource-level — which targets each tool can hit. A query agent gets read access to specific tables, not the whole database.

Action-level — which operations are permitted within each tool. Read and list are near-zero risk. Create is moderate. Update and delete are high-risk and demand additional authorization paths.^[3]

The OWASP Agentic AI Top 10 (December 2025) names this constraint precisely: treat agents as first-class identities with explicit, scoped permissions, use short-lived credentials that rotate frequently, and never allow implicit trust between agents.^[7] The practical corollary is that a shared service account shared between five agents is five attack surfaces pretending to be one.

A dry-run gate is a mandatory preview between proposal and execution. The agent generates a plan — files modified, records affected, infrastructure resources created or destroyed — and a human approves before anything runs.

Terraform got this right with terraform plan. Before any apply, you see exactly what will be created, modified, and destroyed. The bitter joke about Terraform-related agent incidents is that the plan step existed and was either skipped or auto-approved without anyone reading it.^[2] The gate is not the generation. The gate is the read.

For agentic systems, the dry-run gate is mandatory and non-bypassable for any Tier 2 operation. The plan must show:

• What changes (specific files, records, resources)
• How many entities are touched ("3 files" vs "4,200 database rows")
• Whether the change is reversible — and if so, by what mechanism
• What the rollback path actually looks like when something fails

Microsoft's Agent Governance Toolkit (released April 2026, open-source under MIT) implements this pattern as a stateless policy interceptor that runs at sub-millisecond latency (<0.1ms p99) before every tool call.^[9] The same architecture principle applies whether you're using their toolkit or building your own: separate the decision to act from the authorization to execute, with the policy engine as the wall between them.

Deletion protection is the last line — a hard infrastructure constraint that prevents destruction of critical resources regardless of who or what issues the call. This is not access control. It's making certain actions physically impossible without a separate, deliberate process executed by something other than the agent.

AWS ships deletion protection on RDS, DynamoDB, and CloudFormation stacks. GCP has it for Cloud SQL and GKE clusters. Azure has resource locks at the management group level. Terraform has prevent_destroy lifecycle rules. Every major cloud provider arrived at the same conclusion independently: some resources need protection that lives below access controls.^[6] Every production environment running agentic systems should have these flags on before the first agent touches the environment.

The pipeline agent that deleted 18 months of audit logs would have failed at the storage layer if object lock and a retention policy had been enabled. The Terraform agent could not have destroyed the database with prevent_destroy = true set on the resource. These are five-minute configuration changes that close the door on million-dollar incidents.

Blast radius scoring calculates the worst-case impact of an action before it runs. One question, answered before execution: if this fails or behaves outside spec, what is the maximum damage?^[5]

We got this wrong on a live system. An early version of our scoring labeled a config file change "low" — single file, fully versioned, clean rollback path. The file was read by seven services at startup. The change took all seven down on the next rolling restart. The blast radius of the file modification was low. The blast radius of its downstream effects was critical. Dependency mapping is not optional.

Three dimensions decide the score:

Scope — how many resources, records, or systems land in the affected set. An operation touching 5 files is smaller than one touching 5,000 database rows. These thresholds are starting points; calibrate against your workload.

Reversibility — can it be undone, and at what cost. A file modification under version control is fully reversible. A DELETE against a database with no recent backup is effectively irreversible. Irreversible actions are categorically more dangerous regardless of how few entities they touch.

Downstream coupling — what depends on the resources being modified. Changing a shared API contract hits every consumer. Modifying a configuration file hits every service that reads it. The blast radius is not the target. It's the target plus everything coupled to it.

Every agent action — including denials and escalations — gets logged with enough context to reconstruct the incident. The audit log is not compliance theater. It is the forensic instrument that turns a vague production failure into a diagnosable, preventable event.^[3]

A real audit log captures seven fields per action: agent identity, action requested, timestamp, authorization decision (approved, denied, escalated), the dry-run preview that was shown, the execution result, and the rollback status if applicable.

The load-bearing design decision is where the log lives. Outside the agent's reach. If the agent can write to its own audit trail, a malfunctioning agent can erase the evidence of its malfunction during the exact failure mode the log was meant to capture. Append-only storage in a separate account or service. The agent has zero write access. No exceptions.

AI incident investigations without tamper-evident audit trails consistently devolve into blame games. Detailed logs that tie entitlements directly to actions are what separate a 30-minute forensic reconstruction from a three-day postmortem that concludes with "we're not sure what happened."^[10]

The OWASP Agentic Top 10, AWS AgentCore, and Microsoft's open-source Agent Governance Toolkit all arrived at the same architecture independently in 2025 and 2026. That convergence is the signal. The controls are known, the primitives exist on every major cloud, and the incident record is long enough to remove all ambiguity about what happens without them. The only remaining question is whether you build the cage before the agent ships, or after the agent finds the door.