The Production Agent Retirement Checklist

Every conference talk about AI agents covers the launch. Tool calls, context windows, production deployment — the how-to material is everywhere. What got skipped: the end.

Teams that built production agents in 2025 are now making their first retirement decisions. The customer support agent running on a model that has been superseded twice. The internal research tool that accumulated 18 months of edge cases in its system prompt and holds credentials to four internal systems nobody documented. The QA agent that calls three other agents — none of whose owners know they are being invoked.

Shutting these down is messier than it looks. An agent is not just code. It is an identity with credentials, a memory system with accumulated patterns, a service with downstream consumers. "Archive the repo and move on" leaves API keys active, service accounts provisioned, and calling agents broken. It creates what security teams are starting to call ghost agents: retired in intent, live in practice, nobody watching.

This is the playbook that did not get written: treat retiring a production AI agent exactly like decommissioning a microservice. Version the prompt contracts. Run a shadow period. Extract learned patterns before shutdown. Communicate deprecation timelines to internal stakeholders. The structural analogy makes the right behaviors obvious — and skipping any phase has real consequences.

Retirement decisions are harder than launch decisions because the cost of inaction feels low until it does not.

There are five clear triggers worth treating as decision points rather than "we should probably do something" moments:

Performance has structurally degraded. Not a bad week — a persistent trend. The underlying model has been superseded by two generations. Eval scores that looked acceptable six months ago now sit well below the current baseline. When the gap between what the agent does and what a successor could do is wide enough to have business impact, that is a retirement signal, not a tuning task.

The use case has changed. The requirements the agent was built for no longer match what stakeholders actually need. Teams often keep agents running because "it still mostly works," but the definition of "works" has quietly drifted. An agent optimized for a workflow that no longer runs the same way is not an asset — it is maintenance debt with inference costs attached.

The base model is being deprecated. Model providers set end-of-life dates on API endpoints. When the underlying model retires, every agent built on it must either migrate or decommission. Migration means re-evaluating on the new model, not assuming behavior transfers cleanly.

The business context shifted. A product pivot, an acquisition, a process redesign. The workflow this agent was built to accelerate no longer exists in its original form.

Nobody can explain what it does. The most expensive kind. If the person who built it left and no documentation exists, the retirement decision should lean strongly toward shutdown with extraction rather than continued operation with unknown behavior and hidden dependencies.

Software engineering has a well-developed vocabulary for shutting down services: deprecation notices, shadow periods, contract versioning, sunset dates. It has patterns for notifying downstream consumers and handling clients that missed the migration window.

AI agents need the same vocabulary. The parallel is closer than it might appear.

A production agent has a public interface (the prompts it accepts and the outputs it produces), downstream consumers (orchestrators, calling agents, humans relying on its behavior), internal state (memory, embeddings, learned patterns), and credentials granting access to external systems. These map almost exactly to a microservice's API contract, client applications, database state, and service account permissions.

The analogy breaks in one important direction: an agent's "API" — its system prompt — is rarely version-controlled with the rigor applied to an HTTP endpoint. That gap is precisely where retirement goes wrong. Downstream systems that depend on specific output formats, specific tool call patterns, or specific persona behaviors have no interface contract to reference. When the agent disappears, they break silently.

The five-phase retirement pipeline borrows from software engineering practices and adapts them for the specific shape of AI agents. The phases are sequential for a reason. Skipping knowledge extraction before the shadow period means losing institutional memory. Running the hard shutdown before credential revocation means leaving ghost identities with live permissions. The order is not arbitrary — each phase depends on the previous one being complete.

The shadow period is the safest transition tool in the retirement toolkit, and the most frequently skipped.

The mechanism: before the old agent goes dark, the successor runs in parallel. Same inputs, parallel outputs. The successor's responses are collected and logged but not acted upon. You are comparing behavior distributions — not A/B testing user experience.

How to run it: route all production traffic to both agents simultaneously. Collect outputs from both. Run equivalence scoring — semantic similarity over a shared eval set works better than exact string matching. The goal is confirming the successor handles the same failure modes, declines the same request categories, and produces outputs in the expected format and structure.

Shadow periods surface dependencies that no documentation or code review would catch. In one case, a platform team running a shadow period for an invoice processing agent discovered that a downstream orchestrator was extracting a specific field — confidence_score — from the retiring agent's structured output. The successor did not include that field. No automated test had flagged this because the orchestrator degraded gracefully, falling back silently to a default value. The divergence only appeared in the comparison delta between the two outputs.

That is exactly what shadow periods are designed to find: the implicit contracts that never made it into documentation.

Internal stakeholders need a structured deprecation notice. The notice should answer four questions without requiring a follow-up: what is changing, when it stops, what the migration path is, and who to contact with questions.

Borrow from the API deprecation playbook: announce at least 30 days before the shutdown date for anything with integration dependencies. For anything with a hard integration — another agent calling this one, a webhook pointed at this endpoint — 30 days is a minimum. If the shutdown will break a calling agent whose team needs time to update their tool config, 14 days is not enough lead time.

One underappreciated stakeholder: the data and compliance team. They need to approve the data disposition plan before the shutdown begins, not after. Vector stores containing PII have retention and deletion requirements that can extend the retirement timeline. Find this out in Phase 1.

By this point, the successor is running in production, stakeholders have acknowledged the deprecation notice, and the data disposition plan is approved. The hard shutdown itself is the shortest phase. All that remains is execution.

Every item below needs to be completed on the same day the agent stops receiving traffic. The most common mistake: completing the cutover and planning to handle the credential revocation "soon." That gap is how ghost agents are born.

The security dimension of agent retirement gets less attention than the operational dimension, but it is arguably more urgent.

Machine identities — service accounts, API keys, OAuth tokens issued to automated systems — are growing faster than human identities in most enterprise environments ^[3]. The problem: most organizations manage human identity lifecycle carefully — offboarding checklists, IAM reviews, periodic access audits — but apply none of that discipline to machine identities.

An agent that is retired informally — code deleted, repo archived, ticket closed — does not automatically lose its credentials. The service account it used to access the CRM is still provisioned. The API key for the webhook integration is still active. The OAuth token scoped to the data warehouse connector is still valid.

Security teams call these ghost identities: credentials from systems that no longer exist, still live, nobody watching. They are a quiet privilege escalation opportunity. An attacker who finds a valid, scoped API key with read access to customer records does not need to know it belonged to an agent that was retired six months ago. They just need to know it works.

Teams that have run systematic agent audits commonly report finding service accounts provisioned for agents that were never formally retired — live credentials attached to dead systems, sometimes holding permissions that were never scoped down from the original "move fast" deployment. This is not a hypothetical attack vector. It is a cleanup task that compounds with every agent that skips Phase 5.

Gartner projected that 40% of enterprise applications would embed task-specific AI agents by 2026, up from under 5% in 2025 ^[5]. At that scale, informal retirement processes do not just create operational mess — they create a credential sprawl problem that is genuinely difficult to audit retroactively.