# Unprecedented Code Change Automation: The Fusion of LLMs and Transformation by Example

arXiv: https://arxiv.org/abs/2402.07138 · HTML: https://mlsanity.com/paper/2402.07138/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Feb 2024.

## Built on by (most cited first, 15 of 15)

- [You Name It, I Run It: An LLM Agent to Execute Tests of Arbitrary Projects](https://mlsanity.com/paper/2412.10133/) (2024)
- [Bridging the Editing Gap in LLMs: FineEdit for Precise and Targeted Text Modifications](https://mlsanity.com/paper/2502.13358/) (2025)
- [InterTrans: Leveraging Transitive Intermediate Translations to Enhance LLM-based Code Translation](https://mlsanity.com/paper/2411.01063/) (2024)
- [Migrating Code At Scale With LLMs At Google](https://mlsanity.com/paper/2504.09691/) (2025)
- [Artificial-Intelligence Generated Code Considered Harmful: A Road Map for Secure and High-Quality Code Generation](https://mlsanity.com/paper/2409.19182/) (2024)
- [On the Effectiveness of Large Language Models in Writing Alloy Formulas](https://mlsanity.com/paper/2502.15441/) (2025)
- [Treefix: Enabling Execution with a Tree of Prefixes](https://mlsanity.com/paper/2501.12339/) (2025)
- [MigGPT: Harnessing Large Language Models for Automated Migration of Out-of-Tree Linux Kernel Patches Across Versions](https://mlsanity.com/paper/2504.09474/) (2025)
- [OBsmith: LLM-Powered JavaScript Obfuscator Testing](https://mlsanity.com/paper/2510.10066/) (2025)
- [Leveraging LLMs, IDEs, and Semantic Embeddings for Automated Move Method Refactoring](https://mlsanity.com/paper/2503.20934/) (2025)
- [SPELL: Synthesis of Programmatic Edits using LLMs](https://mlsanity.com/paper/2602.01107/) (2026)
- [Multi-Agent Coordinated Rename Refactoring](https://mlsanity.com/paper/2601.00482/) (2026)
- [RepoReviewer: A Local-First Multi-Agent Architecture for Repository-Level Code Review](https://mlsanity.com/paper/2603.16107/) (2026)
- [uGen: An Agentic Framework for Generating Microarchitectural Attack PoCs](https://mlsanity.com/paper/2605.15503/) (2026)
- [Agentic Generation of AST Transformation Rules for Fixing Breaking Updates](https://mlsanity.com/paper/2606.24446/) (2026)

## Abstract (arXiv)

Software developers often repeat code changes, known as "code change patterns" (CPATs), within and across projects. Automating these CPATs accelerates development, but current Transformation by Example (TBE) techniques are limited by the input examples' quality and quantity, missing variations with different syntax or flow yet semantically similar. Large Language Models (LLMs), trained on vast code datasets, can overcome these limitations by generating semantically equivalent, unseen CPAT variants, enhancing TBE effectiveness. We identified best practices for using LLMs to generate code variants meeting criteria of correctness, usefulness, and applicability. Implementing these in PyCraft, combining static and dynamic analysis with LLMs, we achieved an F-measure of 96.6% in identifying correct variants, expanding inputs by 58x on average, and automating changes to increase target codes by up to 39x. Patches from PyCraft were submitted to projects like microsoft/DeepSpeed and IBM/inFairness, with an 83% acceptance rate, validating our approach's usefulness.