# Leveraging LLMs for Grammar Adaptation: A Study on Metamodel-Grammar Co-Evolution

arXiv: https://arxiv.org/abs/2605.21465 · HTML: https://mlsanity.com/paper/2605.21465/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 55 references that are in the map.
- First posted on arXiv 20 May 2026.

## Builds on (most cited first, 12 of 12)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://mlsanity.com/paper/2102.07350/) (2021)
- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [CodeT5+: Open Code Large Language Models for Code Understanding and Generation](https://mlsanity.com/paper/2305.07922/) (2023)
- [An Empirical Study of the Non-determinism of ChatGPT in Code Generation](https://mlsanity.com/paper/2308.02828/) (2023)
- [Pop Quiz! Can a Large Language Model Help With Reverse Engineering?](https://mlsanity.com/paper/2202.01142/) (2022)
- [Towards using Few-Shot Prompt Learning for Automating Model Completion](https://mlsanity.com/paper/2212.03404/) (2022)
- [From Prompt Engineering to Prompt Science With Human in the Loop](https://mlsanity.com/paper/2401.04122/) (2024)
- [Don't Transform the Code, Code the Transforms: Towards Precise Code Rewriting using LLMs](https://mlsanity.com/paper/2410.08806/) (2024)
- [Development and Evolution of Xtext-based DSLs on GitHub: An Empirical Investigation](https://mlsanity.com/paper/2501.19222/) (2025)
- [Leveraging LLMs to support co-evolution between definitions and instances of textual DSLs](https://mlsanity.com/paper/2512.06836/) (2025)
- [Leveraging LLMs to support co-evolution between definitions and instances of textual DSLs: A Systematic Evaluation](https://mlsanity.com/paper/2602.11904/) (2026)

## Abstract (arXiv)

In model-driven engineering, metamodel evolution leads to the need to adapt corresponding grammars to maintain consistency, which typically requires tedious manual work. Existing rule-based methods can achieve partial automation but have limitations when handling complex grammar scenarios. This paper proposes a Large Language Model-based approach that automatically applies adaptations to new grammars after evolution by learning grammar adaptations from previous versions. We evaluated this approach on six real-world Xtext domain-specific languages, using four DSLs as a training set to develop prompting strategies, two DSLs as a test set for validation, and conducting a longitudinal case study on QVTo. The evaluation used three Large Language Models (Claude Sonnet 4.5, ChatGPT 5.1, Gemini 3) and measured grammar adaptation quality from three dimensions: grammar rule-level adaptation consistency, output similarity, and metamodel conformance. Results show that on the test set, all three LLMs achieved 100% adaptation consistency and output similarity, while the rule-based approach achieved only 84.21% on DOT and 62.50% on Xcore. In the QVTo longitudinal study, the LLM-based approach successfully reused learned adaptations across all three evolution steps without manual grammar editing, while the rule-based approach required manual adjustments in two of three transitions. However, on large-scale grammars (EAST-ADL, 297 rules), LLMs' adaptation consistency was far below 90%. This study demonstrates the advantages of LLM-based approaches in handling complex grammar scenarios, while revealing their limitations in large-scale grammar adaptation.