Legacy Codebase Reading Agents: 5 Patterns That Survive the Rewrite

Replacing the old system is the easy part. Reading it is what kills the project.

A payroll engine that clears $200M a year carries thirty years of special cases. A deduction branch added for one customer in 2008. Rounding behavior that downstream reports quietly depend on. A PERFORM call tree nobody has drawn end-to-end since the Bush administration. One engineer still holds the map in his head. He retires in eight months. After that, the org is flying blind into a rewrite it already committed to the board.

The substrate is growing, not retiring. IBM puts active COBOL at 240 billion lines, with five billion added every year^[2]. Median COBOL developer age in the US is 55^[5]. That is not an industry warning. It is a countdown on institutional knowledge nobody bothered to write in a form a system could read.

Most AI-coding commentary assumes greenfield: clean requirements, modern tooling, a team that can grep its own repo. Legacy comprehension is a structurally different problem. The specs disagree with the code. The documentation describes the 1994 design intent. The original engineers retired, left, or died. The rules that move actual money live inside programs written in a language 85% of universities stopped teaching by 1995^[5]. There is no source of truth except the running binary and the source it compiled from.

AI comprehension agents can extract that logic with higher fidelity than a six-month consulting engagement. The phrase set up correctly is doing the load-bearing work in that sentence. The rest of this piece is what correct looks like: five extraction patterns that bind every claim to a source line and reject anything that does not.

Two different problems get collapsed into one, then mishandled together.

A rewrite problem assumes you know what the system does and asks you to rebuild it in a modern language. A comprehension problem assumes the requirements were never written down in any form that survived, the running code is the only source of truth, and the requirements have to be reverse-engineered before anything else is possible. One assumes the answer. The other admits there is no answer yet.

Documentation drift widens the gap further. Every enterprise legacy system was documented at some point. That documentation is now wrong in ways nobody has marked. The comments describe what a developer intended to write in 1998 — not what the code does after fifteen years of patches. The README describes the interface before the 2011 refactor. The wiki flowcharts describe a processing flow superseded by a 2017 hotfix nobody bothered to write up. The rules have to be inferred from the code itself, against no reliable external reference.

This is not a generation problem. It is an extraction problem with an evidence requirement. Every business rule that gets surfaced has to trace back to specific lines in specific files. Anything else is a guess. Confident-sounding fiction is exactly what corrupts the knowledge base on the way to production. The distinction is the whole game.

Every COBOL comprehension pipeline fails or succeeds at the indexer, not the prompt. Most teams get this backward. They spend two days on prompt engineering and twenty minutes on chunking strategy. The model's output quality is then directly proportional to the retrieval quality — which was decided in twenty minutes.

For COBOL and RPG, there are two viable indexing strategies, with sharply different tradeoffs.

AST-based chunking uses a COBOL parser to split source into its syntactic units — divisions, sections, paragraphs — and stores each unit as a retrievable chunk with its full path (program → section → paragraph). The chunk boundaries are semantically meaningful: a paragraph is the smallest coherent unit of COBOL behavior. The retriever can fetch paragraph CALC-DEDUCTIONS from PAYROLL.CBL without any surrounding noise. The failure mode: not all COBOL dialects parse cleanly. IBM OS/VS COBOL, COBOL/400, and older MicroFocus dialects have quirks a generic parser chokes on. You need a parser tuned to your dialect.

LLM-based chunking feeds raw source to a model and asks it to identify coherent behavioral units — effectively asking the LLM to do what an AST parser would do, but without grammar rules. A June 2025 arXiv paper testing this approach on legacy codebases found LLM partitioning is the most viable method when AST parsers aren't available or are prohibitively costly to build for a given dialect. Accuracy is lower than AST parsing — but it works on dialects where no parser exists.^[17]

The third option, and the one most teams default to without realizing it, is fixed-size chunking — split on line count or token count. This is wrong for COBOL. A fixed-size cut in the middle of a PERFORM VARYING loop sends half the loop into one chunk and half into another. The model sees two incomplete fragments instead of one complete behavior. Retrieval quality collapses on exactly the calculation-heavy code that matters most.

Copybook resolution must happen before any chunk hits the index. A COPY CUSTMAST statement in the source is useless without the copybook content. Pre-process every source file to inline-expand all COPY statements, tagging each expanded section with its source copybook name. That tagging is what lets citations trace back to the original copybook — not just to the line in the expanded form.

What separates extraction that ships from extraction that produces plausible-sounding garbage is one discipline: every claim traces to a source line. An uncited rule is a hallucination. It might be a correct hallucination — but with no way to verify, you cannot trust it, which means you cannot ship against it. Each of the five patterns below enforces grounding differently. None of them skip it.

A December 2025 paper on citation-grounded code comprehension demonstrated that hybrid retrieval combining sparse BM25 matching, dense embeddings, and graph-expanded cross-file context outperformed single-mode retrieval by 14–18 percentage points on cross-file architectural queries — exactly the kind of multi-file dependency tracing that COBOL comprehension requires.^[15] The implication: cross-file evidence is where pure text similarity breaks down, and the graph layer is what closes the gap.

The mistake every team makes on their first run: the context dump. They feed the whole COBOL source file — or worse, several files — into the model and ask "what does this do?" The model answers confidently. The answer is wrong in specific, critical places. The model hallucinated behavior for the sections it could not attend to closely, and nobody knows which parts are fabricated.

A July 2025 paper testing multi-agent COBOL explanation found that providing explanations at different granularities of a COBOL project facilitates knowledge transfer and that the multi-agent approach effectively handles both short files and long files that exceed the token window size of LLMs. The key architectural move: a coordinator agent decomposes the program, subagents analyze individual paragraphs in parallel, and a synthesis agent assembles the verified partial results.^[17]

Function-level summarization inverts the topology. Extract a single COBOL paragraph, PERFORM block, or RPG subroutine — the smallest coherent unit of behavior. Retrieve and inline the copybooks it references and the immediate callees it invokes. The model sees exactly enough context to summarize that one unit, with its full dependency chain visible, and nothing else.

The load-bearing discipline: the prompt forces the model to cite the paragraph name, the line range, and every copybook it relied on. A summary without citation cannot be verified, and an unverified summary is indistinguishable from a hallucination. The moment citation becomes a hard requirement, the prompt starts surfacing which summaries the model is confident about and which it is guessing. The guesses stop dressing up as facts.^[9]

Business rule extraction is the highest-upside, highest-failure-rate pattern of the five. The upside: a structured, queryable catalog of what the system actually does, written in plain language a business reviewer can sign off on. The failure mode when done naively: a catalog full of plausible-sounding rules that mix what the code does with what the analyst assumed it does.

The grounding discipline is non-negotiable. Every extracted rule includes the source file name, the line range, and a verbatim snippet of the code that supports the claim. A rule that reads "Premium calculations apply a 12% loading for customers in the high-risk tier" is useless without PREMIUM-CALC.CBL:1203-1241 and the relevant MULTIPLY statement. With the citation, a business analyst validates it in five minutes. Without it, the validation never happens and the rule quietly corrupts the knowledge base.^[10]

Run the extraction twice on the same source. If two runs produce different rules from the same code, the grounding is broken — the model is generating instead of extracting. Treat extraction as a deterministic function: same input, same output. Variance is a hallucination signal, not a quirk.

The December 2025 citation-grounded comprehension research is direct evidence for this design choice: hybrid retrieval with graph expansion achieved 92% citation accuracy with near-zero hallucination rate, compared to 74–79% for pure vector-similarity approaches. The graph layer — tracing which paragraphs CALL which, which copybooks are shared — is what catches cross-file rules the vector retriever misses.^[15]

Data lineage is the most technically demanding extraction pattern and the one with the clearest downstream payoff. Legacy systems rarely process data in a single program. They hand it through chains of COBOL programs, RPG procedures, JCL job steps, and shared copybooks. A field named CUST-BALANCE might be set in ACCT-LOAD.CBL, transformed in BALANCE-ADJ.CBL, fed through a copybook CUSTMAST.CPY used by twelve programs, and finally written to a report by RPT-STMT.CBL. No single developer holds that chain in their head.

A lineage agent builds the graph systematically. Start from a target field — the dollar amount that prints on the customer statement, say — and trace backward. What assigns this field? What reads the assigning program's output? Which copybook defines the shared structure? The agent emits a graph where every edge cites the program and line where the data moves. The output is not a summary. It is a queryable map. "Show me everything that writes CUST-BALANCE before the statement run" becomes answerable without a human tracing CALL statements by hand.^[12]

The December 2025 research confirms the architectural reason this requires graph expansion, not just vector retrieval: cross-file evidence is missed by pure text similarity in 62% of architectural queries. The lineage graph is exactly an architectural query — "how does this field get from program A to program B?" — and vector retrieval alone will miss the transitive dependencies.^[15]

The failure mode that destroys the graph: unresolved copybook expansion. COBOL uses copybooks as shared struct definitions, and the same field name can appear in multiple copybooks with different storage layouts. A lineage agent that traces field references without resolving copybook expansions will produce edges that claim a field is shared between programs when they are actually independent fields that happen to share a name. Resolve all copybooks before any lineage analysis runs. This is not optional.

One pattern separates modernizations that ship from the ones that quietly fail for six months after go-live: generate a behavioral test suite from the legacy system before you write a single line of the new one.

The suite is not documentation. It is a regression harness derived from observing what the legacy system actually does with real inputs. Run the extraction agent across the codebase to identify the significant behavioral branches. Instrument the legacy system to capture input/output pairs for each branch — ideally from production traffic, or from a test data set built to cover the branches the agent identified. Generate test cases that assert the observed behavior. Then run those cases against the legacy system to confirm they are right. The suite goes green against the thing being replaced.

AWS launched automated test plan generation for mainframe in December 2025, specifically targeting this problem: generating test data collection scripts for VSAM, GDG, PDS, and DB2 sources and producing functional test scripts from the resulting test plan. AWS's own assessment is that testing typically consumes over 50% of mainframe modernization project duration — the behavioral test generation step is the single largest leverage point for compressing that.^[16]

Now correctness has a definition that does not depend on anyone's memory or on documentation that may be wrong. When the new system passes the suite, the team has evidence — not faith — that it behaves equivalently. When it fails, the divergence is precise: a specific input and a specific output difference, not a vague "something seems off" from a user in UAT.^[9]

Teams that skip this step discover their requirements during production incidents. The deduction branch that only fires for one customer processes $4M a year. It was in no spec. It was in no test the new system carried. It shows up three months after cutover in a payroll discrepancy.

The strangler fig pattern — routing new traffic to a modern service while the legacy system handles the rest — depends on a clean interface definition between the two. That interface cannot be guessed or invented. It has to match what the legacy program actually accepts and returns, or the routing layer becomes a translation layer that accumulates its own bugs.

Interface synthesis extracts the API contract directly from the legacy program's entry points: the LINKAGE SECTION in COBOL, the parameter lists in RPG procedures, the method signatures in legacy Java. The agent maps every parameter to its copybook definition or class field, preserves original data types and lengths, and emits an OpenAPI specification (or equivalent) that drives the modern service's contract. Every field in the spec traces back to a named field in the legacy source. No invented names. No guessed types. The strangler fig router is built against this spec, not against an assumption.^[6]

Concretely: a COBOL batch program with a LINKAGE SECTION defining CUST-IN (PIC X(8)), AMOUNT (PIC S9(11)V99 COMP-3), and RETURN-CODE (PIC S9(4) COMP) maps to an OpenAPI schema with custId (string, maxLength: 8), amount (number, format: packed-decimal, precision: 2), and returnCode (integer). The packed-decimal type annotation is not cosmetic — it determines how the modern service marshals data for comparison runs during the coexistence phase. Get it wrong and the parallel-run parity check fails silently.

This is the shortest path from legacy comprehension to active migration. Once the interface spec is grounded, the new service can stand up and start absorbing a subset of traffic while the legacy system handles the rest — the coexistence playbook in practice.

The tool market for legacy comprehension expanded sharply through 2025 and 2026. Every major cloud vendor has a product. IBM has a mainframe-specific offering. A new category of code intelligence platforms has shown up. Honest read: most of these tools help with the comprehension problem. None of them solve it without a grounding discipline imposed by the team. The tool is a retrieval and generation layer. The grounding is the team's responsibility.

In February 2026, Anthropic published a technical position on COBOL modernization, arguing that Claude Code can map dependencies across thousands of lines of code, trace execution paths through called subroutines, and document cross-module dependencies that static analysis tools miss. The counter from IBM: decades of hardware-software integration cannot be replicated by moving code. Both claims are partially right. Claude Code handles the comprehension and documentation phase well — it does not handle the IBM Z systems software dependencies that Watsonx was trained on.^[14]

The best-performing teams in this space were not using the most sophisticated tool. They were running a general-purpose model with a rigorous prompting discipline and a human validation step that rejected anything without a source citation. The tool ceiling matters less than the process floor. Buying Watsonx and abandoning the grounding discipline produces worse output than running Claude Code with strict citation requirements.

Modernizations fail because the system gets rewritten before it gets understood. Confidence fills the comprehension gap — developer confidence, consultant confidence, project manager confidence that the legacy system is simpler than it looks. It is never simpler than it looks. Thirty years of business logic, edge cases, and undocumented behavior accumulated in that COBOL because the business kept changing and the system kept being patched to match.

The tools are good enough. Anthropic published a COBOL modernization playbook in February 2026. AWS shipped mainframe test automation in December 2025. IBM has had mainframe-specific AI assistance in production for two years. The constraint is not the tool — it's the discipline to use any of them correctly.^[14][16] Read first. Map the rules. Generate the tests. Then decide whether to rewrite.

Ground every claim. Validate every rule. A fast extraction is not the same thing as a correct one, and the system being replaced cannot tell you the difference.