# Agentic Method for Deterministic Validation of Legacy Code Migration

arXiv: https://arxiv.org/abs/2607.28271 · HTML: https://mlsanity.com/paper/2607.28271/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 22 of its 37 references that are in the map.
- First posted on arXiv 30 Jul 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 22 of 22)

- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [Illuminating search spaces by mapping elites](https://mlsanity.com/paper/1504.04909/) (2015)
- [RepairAgent: An Autonomous, LLM-Based Agent for Program Repair](https://mlsanity.com/paper/2403.17134/) (2024)
- [An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation](https://mlsanity.com/paper/2302.06527/) (2023)
- [Automated Unit Test Improvement using Large Language Models at Meta](https://mlsanity.com/paper/2402.09171/) (2024)
- [The Art, Science, and Engineering of Fuzzing: A Survey](https://mlsanity.com/paper/1812.00140/) (2018)
- [A Survey of Learning-based Automated Program Repair](https://mlsanity.com/paper/2301.03270/) (2023)
- [A Survey of Symbolic Execution Techniques](https://mlsanity.com/paper/1610.00502/) (2016)
- [A Survey of LLM-based Automated Program Repair: Taxonomies, Design Paradigms, and Applications](https://mlsanity.com/paper/2506.23749/) (2025)
- [Monte Carlo Elites: Quality-Diversity Selection as a Multi-Armed Bandit Problem](https://mlsanity.com/paper/2104.08781/) (2021)
- [An Empirical Study of OSS-Fuzz Bugs](https://mlsanity.com/paper/2103.11518/) (2021)
- [A survey on software testability](https://mlsanity.com/paper/1801.02201/) (2018)
- [Mokav: Execution-driven Differential Testing with LLMs](https://mlsanity.com/paper/2406.10375/) (2024)
- [XMainframe: A Large Language Model for Mainframe Modernization](https://mlsanity.com/paper/2408.04660/) (2024)
- [DiffSpec: Differential Testing with LLMs using Natural Language Specifications and Code Artifacts](https://mlsanity.com/paper/2410.04249/) (2024)
- [Automated Testing of COBOL to Java Transformation](https://mlsanity.com/paper/2504.10548/) (2025)
- [A Snowballing Literature Study on Test Amplification](https://mlsanity.com/paper/1705.10692/) (2017)
- [Mut4All: Fuzzing Compilers via LLM-Synthesized Mutators Learned from Bug Reports](https://mlsanity.com/paper/2507.19275/) (2025)
- [Automated Validation of COBOL to Java Transformation](https://mlsanity.com/paper/2506.10999/) (2025)
- [Automatic Test Improvement with DSpot: a Study with Ten Mature Open-Source Projects](https://mlsanity.com/paper/1811.08330/) (2018)
- [COBOL-Coder: Domain-Adapted Large Language Models for COBOL Code Generation and Translation](https://mlsanity.com/paper/2604.03986/) (2026)

## Abstract (arXiv)

Migration of legacy COBOL programs to Java requires extensive testing to ensure correct functionality. This effort is often complicated by the lack of test data and the difficulty of validating all corner cases. In this paper we propose a novel agentic test-synthesis method, the "Locksmith Loop," which is initiated by preparing two runtime environments: the COBOL source and the generated Java target are each instrumented with mocks and executed off-mainframe on commodity hardware, then an iterative agentic loop performs Witness Search over input mocks to penetrate program branches, followed by parity-preserving mutations. When routing boundaries are reached, an analyzer identifies a Locked Paragraph: a condition preventing deeper exploration. Across three COBOL-Java case studies, spanning two open-source programs and one internal production-like COBOL program and ranging from 430 to 4,114 source lines, Locksmith consistently improved coverage beyond input-search plateaus, reaching nearly complete coverage on the two open-source programs and 91.90% branch coverage on the internal production-like COBOL program. The generated Java matched the COBOL reference under deterministic parity checks in all accepted test cases. Through these findings we demonstrate, to the best of our knowledge, a novel approach for validating agentic coding output using a deterministic oracle.