# CoSTAR: Data Synthesis-Driven Constraint-Aware COBOL Section Summarization for Legacy System Modernization

arXiv: https://arxiv.org/abs/2609.11332 · HTML: https://mlsanity.com/paper/2609.11332/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Code and instruction data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 15 of its 53 references that are in the map.
- First posted on arXiv 10 Sep 2026.

## Placed in

- Synthetic data and self-improvement for LLMs / [Code and instruction data synthesis](https://mlsanity.com/synth/code-and-instruction-data-synthesis/)

## Builds on (most cited first, 15 of 15)

- [Few-shot training LLMs for project-specific code-summarization](https://mlsanity.com/paper/2207.04237/) (2022)
- [Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization)](https://mlsanity.com/paper/2304.06815/) (2023)
- [Multilingual training for Software Engineering](https://mlsanity.com/paper/2112.02043/) (2021)
- [Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions](https://mlsanity.com/paper/2308.02312/) (2023)
- [SPT-Code: Sequence-to-Sequence Pre-Training for Learning Source Code Representations](https://mlsanity.com/paper/2201.01549/) (2022)
- [Distilled GPT for Source Code Summarization](https://mlsanity.com/paper/2308.14731/) (2023)
- [Source Code Summarization in the Era of Large Language Models](https://mlsanity.com/paper/2407.07959/) (2024)
- [AlphaTrans: A Neuro-Symbolic Compositional Approach for Repository-Level Code Translation and Validation](https://mlsanity.com/paper/2410.24117/) (2024)
- [CodeFlowBench: A Multi-turn, Iterative Benchmark for Complex Code Generation](https://mlsanity.com/paper/2504.21751/) (2025)
- [Code Needs Comments: Enhancing Code LLMs with Comment Augmentation](https://mlsanity.com/paper/2402.13013/) (2024)
- [Can Large Language Models Serve as Evaluators for Code Summarization?](https://mlsanity.com/paper/2412.01333/) (2024)
- [ESALE: Enhancing Code-Summary Alignment Learning for Source Code Summarization](https://mlsanity.com/paper/2407.01646/) (2024)
- [XMainframe: A Large Language Model for Mainframe Modernization](https://mlsanity.com/paper/2408.04660/) (2024)
- [Leveraging LLMs for Legacy Code Modernization: Challenges and Opportunities for LLM-Generated Documentation](https://mlsanity.com/paper/2411.14971/) (2024)
- [Context-aware Code Summary Generation](https://mlsanity.com/paper/2408.09006/) (2024)

## Abstract (arXiv)

COBOL remains critical to governments, financial institutions, and large enterprises; yet, aging technologies, shrinking expertise, and missing documentation make modernization of COBOL-based legacy systems increasingly urgent. Before migration, code summarization is a common practice to support legacy system understanding. However, COBOL code summarization, especially on section-level, faces two key challenges: data scarcity and migration constraint preservation. To address these challenges, we propose CoSTAR, an integrated framework that combines execution-validated data synthesis with constraint-aware model training. CoSTAR repurposes general-purpose programming tasks to synthesize execution-validated COBOL code-summary data through LLM-based generation to overcome data scarcity. Based on the synthesized data, CoSTAR augments target sections with relevant data declarations and natural-language explanations, and uses constraint-guided structured rationales to train smaller base LLMs. The trained LLMs preserve the migration constraints for COBOL section summarization. We evaluate CoSTAR on both public and confidential enterprise COBOL systems. CoSTAR effectively synthesizes 3,764 execution-validated training instances. Based on these instances, CoSTAR built on 7B/8B base LLMs can improve these LLMs with average relative gains of 25.38% on ROUGE-L, 53.84% on METEOR, and 37.22% on chrF. In real-world enterprise evaluation, CoSTAR built on only Qwen3-8B, outperforms the enterprise-deployed Qwen3-235B in accuracy, completeness, and conciseness. These results show that CoSTAR enables small, locally deployable LLMs to achieve performance competitive with substantially larger LLMs for privacy-sensitive COBOL legacy systems.