On February 23, 2026, IBM’s share price fell 13 percent in a single day, its steepest one-day decline since October 2000. The trigger was an announcement from an AI lab claiming the ability to modernize COBOL. Boards with mainframe estates read that announcement and asked whether AI could now convert their COBOL and let them switch the mainframe off. IBM’s answer, published the same day, was precise: translation captures almost none of the real complexity, which sits in data architecture, runtime replacement, transaction integrity, and the non-functional requirements built into the platform. The market had confused two different problems and priced a solution to one as if it were a solution to both.
The two problems are comprehension and translation. They are not phases of the same activity. They are genuinely different technical challenges that AI handles with genuinely different capability and genuinely different risk. AI has made the COBOL estate dramatically cheaper to understand. It has not made it dramatically cheaper to replace. The distinction matters precisely because organizations making modernization investment decisions need to know which problem they are paying to solve, what AI actually delivers on each, and where the verification work that remains regardless of AI tooling will consume the budget that comprehension and translation savings freed up.
The Structural Foundation AI Needs
SMART TS XL identifies unreachable programs before AI translation starts — so you translate what runs, not what exists.
FIND OUT MORE…The Distinction That the Market Got Wrong
Comprehension and translation are not the same problem.
Comprehension is the discipline of understanding what code does and why, extracting the business intent from the technical implementation. In a COBOL context, comprehension means: what business rules does this program encode? What conditions govern each execution path? What data does it read and produce? What downstream systems depend on its outputs? What would a business analyst need to know about this program to write a specification for its replacement? Comprehension is about semantics, meaning, not syntax.
Translation is the discipline of converting code from one language to another while preserving its behavior. In a COBOL context, translation means: produce Java (or Python, or Go) that does what the COBOL does, for all inputs, under all conditions, with numerical equivalence sufficient for the downstream systems that consume the outputs. Translation is about structural and behavioral equivalence, the new code must pass every test the old code would have passed, including tests for edge cases the original developer considered and you may never have known existed.
| Dimension | Comprehension | Translation |
|---|---|---|
| Question answered | What does this code mean? | What does this code do, exactly? |
| Output type | Natural language, documentation | Executable code |
| Error consequence | Correctable by human review | Production incident |
| AI capability | Strong, LLMs excel at prose | Weak, determinism required |
| Scale constraint | Manageable with context windows | Portfolio-scale fails context limit |
| Risk profile | Low, errors are reviewable | High, errors are invisible until they fire |
| Human role | Validator and enricher | Reviewer and prover |
These problems are related, you need some comprehension to do good translation, but they are not the same problem, and AI’s capability gap between them is significant. AI comprehension of COBOL code is genuinely impressive. AI translation of COBOL code to production-correct Java is genuinely difficult, and the market’s conflation of the two is why Gartner predicted that more than 70 percent of mainframe exit projects initiated in 2026 will fail to produce their intended benefits.
Where AI Comprehension Actually Delivers
The comprehension problem has a characteristic that makes AI well-suited to it: the output is natural language, and natural language generation is what large language models do well. When AI reads a COBOL program and produces documentation, a description of what the program does, an explanation of its business rules, an enumeration of its data flows, the output is text. Errors in that text are recoverable: a human reviewer can catch a mischaracterized business rule and correct it without any downstream technical consequence. The cost of comprehension errors is a documentation correction, not a production incident.
Business rule documentation. A large language model presented with a COBOL EVALUATE block, potentially hundreds of WHEN conditions accumulated over decades of regulatory changes, can produce a structured description of what each condition means in business terms. For programs where the original documentation no longer exists or never existed, this is genuinely valuable. A COBOL program that implements tax calculation logic across 63 jurisdictions has those jurisdiction rules encoded in its EVALUATE structure; AI can extract and document them faster than any manual process. What it produces should be validated by a domain expert, but the expert is validating AI output rather than reconstructing knowledge from scratch, which is a much faster process.
Here is what AI comprehension looks like in practice. Given this COBOL source:
cobol
EVALUATE TRUE
WHEN CUST-COUNTRY = 'US' AND CUST-STATE = 'CA'
COMPUTE TAX-AMT = GROSS-AMT * 0.0725
WHEN CUST-COUNTRY = 'US' AND CUST-STATE = 'TX'
COMPUTE TAX-AMT = GROSS-AMT * 0.0825
WHEN CUST-COUNTRY = 'UK'
COMPUTE TAX-AMT = GROSS-AMT * 0.20
WHEN CUST-COUNTRY = 'DE'
COMPUTE TAX-AMT = GROSS-AMT * 0.19
WHEN OTHER
MOVE ZERO TO TAX-AMT
END-EVALUATE.
A well-prompted LLM produces:
Business Rule: Tax Calculation
This paragraph applies jurisdiction-specific tax rates to GROSS-AMT:
- US/California: 7.25% (CA state sales tax)
- US/Texas: 8.25% (TX state sales tax)
- United Kingdom: 20% (UK VAT standard rate)
- Germany: 19% (DE MwSt standard rate)
- All other countries: tax-exempt (TAX-AMT = 0)
Note: No rate exists for other US states. Orders from NY, FL, IL
will be tax-exempt under current logic -- likely a gap requiring review.That output is genuinely valuable: it documents four business rules, identifies a likely gap (US states without explicit rates), and does so in hours. A domain expert reviews it, confirms the gap, and adds it to the test cases. The comprehension task completed correctly.
Dependency narrative. Static analysis tools produce dependency graphs, which programs call which, which copybooks are shared, which datasets flow between programs. AI can produce a natural language narrative from that graph: “CUSTPROC is called by 14 programs, primarily in the nightly batch window. It shares CUSTMSTR with 47 programs and produces output to CUST-REPORT-FILE, which is consumed by the regulatory reporting workflow.” This narrative is more useful for onboarding and architecture review than a dependency graph alone, and AI produces it in seconds from the structured dependency data.
Test case generation from understood behavior. Once AI has documented the program’s behavior, it can generate test cases that cover the documented execution paths. This is not the same as proving behavioral equivalence, the test cases cover paths the AI understood, which may miss paths it did not. But covering 60 percent of execution paths through AI-generated tests is better than the zero percent coverage that many legacy programs currently have, and it provides a starting point for the manual test augmentation that behavioral equivalence validation requires.
Pattern recognition across similar programs. Large COBOL portfolios frequently contain many programs that are structurally similar, the same processing pattern applied to different entities, the same regulatory calculation parameterized differently. AI can identify these similarities across thousands of programs and cluster them by pattern type, enabling a migration strategy that addresses each pattern once and applies the solution systematically rather than treating every program as unique.
The common thread: AI comprehension outputs are text, text is reviewable by humans, and human review catches errors before they have technical consequences. The risk profile of AI comprehension is low enough that the value it delivers is genuine and the errors it makes are manageable.
Where AI Translation Overpromises
Translation has a characteristic that makes it fundamentally harder for AI than comprehension: the output is executable code, and executable code must be exactly right. A 95 percent accurate translation produces a program that fails for 5 percent of inputs, which in a financial transaction processing system may represent a regulatory reporting error, a balance calculation discrepancy, or a compliance finding. The cost of translation errors is not a documentation correction. It is a production incident.
The numeric precision failure. COBOL’s packed decimal arithmetic (COMP-3) produces exact decimal results. Java’s double and Python’s float use IEEE 754 binary floating-point, which cannot exactly represent most decimal fractions. A COBOL program that calculates 0.1 + 0.2 with COMP-3 produces exactly 0.3. The same calculation in Java with double produces 0.30000000000000004. For a financial institution where this calculation runs 10 million times per night across regulatory submissions, the accumulated error is not a floating-point curiosity, it is a financial reporting discrepancy. An AI translation that generates Java without explicitly handling the COMP-3-to-BigDecimal mapping introduces this error systematically, and it does so for every field the AI translates as double rather than BigDecimal.
The difference in code is subtle; the consequence is not:
cobol
* COBOL: COMP-3 packed decimal -- exact arithmetic
77 UNIT-PRICE PIC S9(7)V99 COMP-3.
77 QUANTITY PIC S9(5) COMP-3.
77 LINE-TOTAL PIC S9(11)V99 COMP-3.
COMPUTE LINE-TOTAL = UNIT-PRICE * QUANTITY.
* UNIT-PRICE = 19.99, QUANTITY = 3
* LINE-TOTAL = exactly 59.97 -- no rounding error
java
// Java: naive AI translation uses double -- introduces error
double unitPrice = 19.99;
int quantity = 3;
double lineTotal = unitPrice * quantity;
// lineTotal = 59.96999999999999... not 59.97
// Correct translation requires BigDecimal:
// BigDecimal lineTotal = new BigDecimal("19.99").multiply(new BigDecimal(3));At one transaction the error is invisible. At 10 million nightly transactions, summed into a regulatory capital report, it is a material discrepancy. An AI translation that does not explicitly address COMP-3 fields will make this substitution every time.
The compiler option dependency. Identical COBOL source code can produce different behavior under different compiler options. NUMPROC(MIG) vs NUMPROC(NOPFD) changes how numeric exceptions are handled. RMODE, AMODE, and DATA options affect addressing. An AI that translates COBOL source without knowing which compiler options were in use translates the source of the program, not the behavior of the program. The translated Java may match the source but not the production behavior.
The hallucination problem at scale. Research shows GPT-4-class models exhibit 33 to 60 percent hallucination rates in general code translation tasks, generating plausible-looking code that adds or removes behavior not present in the source. For a program with 847 EVALUATE conditions, a 5 percent hallucination rate means approximately 42 conditions are incorrectly translated. Those 42 conditions represent business rules that the translation silently broke. Without a complete test corpus that exercises every condition, those breaks are undetectable until the specific input combination that triggers them reaches production.
The stakes in numbers:
Error Type | Detection Point | Cost Range
--------------------|--------------------|------------------
Comprehension error | Human review | Hours to fix
Translation error | Test suite | Days to isolate
Translation error | Production | $10K-$500K+ incident
Equivalence gap | Regulatory audit | $50K-$5M+ penaltyThe asymmetry: Comprehension errors are caught before they have technical consequences. Translation errors survive review and detonate in production, or worse, in a regulatory submission.
The context window ceiling. IBM’s watsonx Code Assistant for Z achieves 80 percent or better structural accuracy on COBOL-to-Java translation for individual programs within a context window. A 5-million-line COBOL portfolio consists of thousands of programs, with cross-program dependencies that span programs that cannot all be in the same context window simultaneously. A program that calls a subprogram whose interface the AI translated differently than the calling program’s assumptions creates an interface mismatch that neither translation alone would reveal. Portfolio-scale translation requires understanding across programs, not just within them.
The context-free translation problem. A COBOL program that processes a specific business event does so with context that is embedded in the system around it, the batch schedule that determines when it runs, the datasets that must be available when it starts, the order in which it processes records, the downstream programs that read its output before they run. An AI that translates the program without understanding this context translates the source but not the role. The translated Java may be internally correct and still fail operationally because it runs at the wrong time, in the wrong order, without the data prerequisites its COBOL predecessor relied on.
The Behavioral Equivalence Gap
The most significant problem that neither comprehension AI nor translation AI solves is behavioral equivalence proof, the demonstration that the modernized system produces the same outputs as the legacy system for all inputs that matter to the business and its regulators.
Behavioral equivalence is not a testing problem in the standard sense. It is not asking “does the new system pass the test suite?”, because the test suite for a legacy COBOL system often does not exist, or covers a fraction of the execution paths the program contains. It is asking: for every input the production system has ever received, does the modernized system produce the same output? And for every input the production system might receive in the future, including edge cases that occur only during rare market conditions, regulatory reporting periods, or specific customer circumstances, does the modernized system produce the equivalent result?
Answering this question requires: a characterization of the existing system’s behavior derived from actual production execution (not from documentation, not from the source alone), a test corpus that exercises the full range of that behavior, and a comparison framework that can adjudicate what “equivalent” means for outputs where the representation differs (EBCDIC vs UTF-8, packed decimal vs BigDecimal, batch file vs database record).
None of this is what AI translation tools provide. IBM’s assessment is direct: translation captures almost none of the real complexity of modernization. The behavioral equivalence work, the parallel run, the reconciliation framework, the regulatory sign-off, is where the cost and risk concentrate, and it is where AI makes the least difference to the total program cost. AI compresses the front of the modernization project: comprehension and initial translation. It does not compress the back: verification, equivalence testing, and parallel validation.
The Correct Model: Static Analysis Foundation, AI Semantic Layer
The most effective AI comprehension capability is not AI alone, it is AI operating on top of a structural foundation that static analysis provides. The limitations of AI comprehension (context window limits, hallucination on complex cross-program logic, probabilistic rather than deterministic analysis) are addressed by static analysis. The limitations of static analysis (it produces structural facts but not semantic descriptions) are addressed by AI.
The right division of labor: Static analysis provides structural certainty (deterministic, complete, no hallucinations). AI provides semantic richness (natural language descriptions, pattern recognition, test generation). Neither alone is sufficient. Together they cover the full comprehension requirement.
The structural foundation that static analysis provides:
Complete program inventory. Every program in the portfolio, including programs that documentation missed and programs that AI’s context window would never hold simultaneously. Static analysis processes the full portfolio without context limits.
Accurate dependency graph. Every CALL relationship, every COPY relationship, every data flow between programs, derived deterministically from the source, not probabilistically from training data. When an AI comprehension tool describes a program’s dependencies, that description should be grounded in a dependency graph that static analysis produced, not in the AI’s probabilistic recall of what programs typically depend on.
Dead code identification. Programs unreachable from any production execution path are not modernization scope. Static analysis identifies them with certainty; AI cannot, because AI does not have the runtime execution traces necessary to distinguish live code from dead code by static analysis alone.
Data schema inventory. Every FD entry, every copybook, every COMP-3 field with its precision specification, extracted from source artifacts rather than reconstructed from the AI’s training-data memory of what COBOL field definitions typically look like.
On top of this structural foundation, AI comprehension adds the semantic layer: the natural language descriptions of what each program does, the business rule documentation, the narrative of cross-program data flows, the pattern recognition that clusters similar programs. This combination, deterministic structural accuracy from static analysis, natural language semantic richness from AI, is the model that produces comprehension outputs a modernization team can actually rely on.
What This Means for Your Modernization Program
Practical allocation of AI investment based on this analysis:
Invest in AI comprehension. Documentation generation, business rule extraction, dependency narrative, test case generation from documented behavior, these are high-value, low-risk uses of AI that compress the most expensive traditionally manual phase of modernization. Do these first, before translation, because comprehension informs every subsequent decision.
Use AI translation as a draft generator, not a finalizer. AI translation produces a Java or Python starting point that a developer reviews, corrects, and validates, not production-ready code that can be deployed without human review. Treating AI translation output as a draft reduces the cost of translation while maintaining the human accountability that equivalence validation requires.
Do not count AI translation savings against verification cost. AI compresses the time to produce a translated program. It does not compress the time to prove that the translated program is equivalent to the original. Budget the verification phase independently of AI translation savings, because the verification scope does not shrink when translation gets faster.
Build the structural foundation first. AI comprehension tools that lack a structural foundation, an accurate dependency graph, a complete program inventory, a data schema inventory, produce comprehension that is incomplete where the code is most complex. The programs that most need AI comprehension are typically the most complex, most cross-program-dependent, and most likely to exceed what AI can accurately analyze from source alone.
How SMART TS XL Supports AI-Enhanced COBOL Comprehension
SMART TS XL provides the structural analysis foundation that makes AI comprehension of COBOL portfolios accurate rather than probabilistic.
The static code analysis capability produces the program-level structural data that AI comprehension tools need: every EVALUATE condition enumerated, every COMPUTE formula extracted, every data access pattern characterized. When AI documentation tools operate on source code with this structural data as context, they produce descriptions grounded in what the code actually contains rather than in what similar-looking programs typically do.
The application dependency mapping provides the cross-program context that AI context windows cannot hold simultaneously. A dependency graph that shows CUSTPROC’s 14 callers, its 47 copybook sharers, and its 3 data flow outputs gives AI comprehension tools the full picture of a program’s role in the portfolio, without requiring every dependent program to be in the same context window at the same time.
The legacy modernization analysis produces the dead code inventory and complexity distribution that AI comprehension cannot reliably produce alone: which programs are unreachable from production (excluded from scope), which programs have the highest complexity (where comprehension errors are most expensive), and what the data migration scope looks like derived from actual FD entries rather than from documentation.
The enterprise search capability makes the structural inventory queryable in support of AI-assisted comprehension: find every program that implements a specific pattern, every EVALUATE block with more than N conditions, every program that accesses a specific dataset, queries that ground AI comprehension in structural evidence rather than in training-data probability.
For organizations where AI is now part of the modernization toolchain, SMART TS XL’s structural analysis is the layer that makes AI comprehension accurate at portfolio scale, addressing the context window limit, the hallucination risk on cross-program logic, and the determinism gap that separates structural analysis from probabilistic generation.
AI Changed the Cost of Understanding. Not the Cost of Proving.
What changed in 2026 is the cost of comprehension. AI has made it dramatically cheaper and faster to understand what a COBOL program does, document its business rules, characterize its behavior, and describe its place in the portfolio. This is a genuine and substantial contribution to modernization economics. For programs where the comprehension phase previously required weeks of specialist effort, AI can produce a draft understanding in hours that specialists review and refine rather than construct from scratch.
What did not change is the cost of proving. The behavioral equivalence validation, the parallel run, the regulatory sign-off, the data migration verification, these remain where they were before AI, consuming 40 to 50 percent of modernization budget regardless of how fast the comprehension and translation phases complete. IBM was right on February 23, 2026: translation captures almost none of the real complexity. The real complexity is in the verification of behavioral equivalence, and that complexity does not compress when the translation gets faster.
The organizations that use AI well in COBOL modernization are the ones that invest in AI comprehension, where AI is genuinely excellent, and treat AI translation as a starting point for human review rather than a finished product ready for deployment. They understand that the front of the project getting cheaper does not make the back of the project cheaper. And they build the structural analysis foundation that AI comprehension needs to be accurate at the portfolio scale where the most important comprehension work happens.
