# MANTRA: Enhancing Automated Method-Level Refactoring with Contextual RAG and Multi-Agent LLM Collaboration

arXiv: https://arxiv.org/abs/2503.14340 · HTML: https://mlsanity.com/paper/2503.14340/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 18 of its 67 references that are in the map.
- First posted on arXiv 18 Mar 2025.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 18 of 18)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [CodeBLEU: a Method for Automatic Evaluation of Code Synthesis](https://mlsanity.com/paper/2009.10297/) (2020)
- [ChatGPT Prompt Patterns for Improving Code Quality, Refactoring, Requirements Elicitation, and Software Design](https://mlsanity.com/paper/2303.07839/) (2023)
- [AI-assisted coding: Experiments with GPT-4](https://mlsanity.com/paper/2304.13187/) (2023)
- [SOEN-101: Code Generation by Emulating Software Process Models Using Large Language Model Agents](https://mlsanity.com/paper/2403.15852/) (2024)
- [Refactoring Programs Using Large Language Models with Few-Shot Examples](https://mlsanity.com/paper/2311.11690/) (2023)
- [How to Refactor this Code? An Exploratory Study on Developer-ChatGPT Refactoring Conversations](https://mlsanity.com/paper/2402.06013/) (2024)
- [Why We Refactor? Confessions of GitHub Contributors](https://mlsanity.com/paper/1607.02459/) (2016)
- [LLM4EFFI: Leveraging Large Language Models to Enhance Code Efficiency and Correctness](https://mlsanity.com/paper/2502.18489/) (2025)
- [EM-Assist: Safe Automated ExtractMethod Refactoring with LLMs](https://mlsanity.com/paper/2405.20551/) (2024)
- [On the Relationship between Refactoring Actions and Bugs: A Differentiated Replication](https://mlsanity.com/paper/2009.11685/) (2020)
- [A Novel Refactoring and Semantic Aware Abstract Syntax Tree Differencing Tool and a Benchmark for Evaluating the Accuracy of Diff Tools](https://mlsanity.com/paper/2403.05939/) (2024)
- [Refactoring-aware Block Tracking in Commit History](https://mlsanity.com/paper/2409.16185/) (2024)
- [One Thousand and One Stories: A Large-Scale Survey of Software Refactoring](https://mlsanity.com/paper/2107.07357/) (2021)

## Built on by (most cited first, 10 of 10)

- [RobuNFR: Evaluating the Robustness of Large Language Models on Non-Functional Requirements Aware Code Generation](https://mlsanity.com/paper/2503.22851/) (2025)
- [SkillScope: Toward Fine-Grained Least-Privilege Enforcement for Agent Skills](https://mlsanity.com/paper/2605.05868/) (2026)
- [SmellBench: Evaluating LLM Agents on Architectural Code Smell Repair](https://mlsanity.com/paper/2605.07001/) (2026)
- [Code for Machines, Not Just Humans: Quantifying AI-Friendliness with Code Health Metrics](https://mlsanity.com/paper/2601.02200/) (2026)
- [Leveraging LLMs, IDEs, and Semantic Embeddings for Automated Move Method Refactoring](https://mlsanity.com/paper/2503.20934/) (2025)
- [A Blueprint for AI-Driven Software Quality: Integrating LLMs with Established Standards](https://mlsanity.com/paper/2505.13766/) (2025)
- [BabelCoder: Agentic Code Translation with Specification Alignment](https://mlsanity.com/paper/2512.06902/) (2025)
- [Multi-Agent Coordinated Rename Refactoring](https://mlsanity.com/paper/2601.00482/) (2026)
- [AI-Assisted Unit Test Writing and Test-Driven Code Refactoring: A Case Study](https://mlsanity.com/paper/2604.03135/) (2026)
- [From Blueprint to Reality: Modeling and Applying Putnam's Social Capital Theory with LLM-based Multi-agent Simulations](https://mlsanity.com/paper/2607.06080/) (2026)

## Abstract (arXiv)

Maintaining and scaling software systems relies heavily on effective code refactoring, yet this process remains labor-intensive, requiring developers to carefully analyze existing codebases and prevent the introduction of new defects. Although recent advancements have leveraged Large Language Models (LLMs) to automate refactoring tasks, current solutions are constrained in scope and lack mechanisms to guarantee code compilability and successful test execution. In this work, we introduce MANTRA, a comprehensive LLM agent-based framework that automates method-level refactoring. MANTRA integrates Context-Aware Retrieval-Augmented Generation, coordinated Multi-Agent Collaboration, and Verbal Reinforcement Learning to emulate human decision-making during refactoring while preserving code correctness and readability. Our empirical study, conducted on 703 instances of "pure refactorings" (i.e., code changes exclusively involving structural improvements), drawn from 10 representative Java projects, covers the six most prevalent refactoring operations. Experimental results demonstrate that MANTRA substantially surpasses a baseline LLM model (RawGPT ), achieving an 82.8% success rate (582/703) in producing code that compiles and passes all tests, compared to just 8.7% (61/703) with RawGPT. Moreover, in comparison to IntelliJ's LLM-powered refactoring tool (EM-Assist), MANTRA exhibits a 50% improvement in generating Extract Method transformations. A usability study involving 37 professional developers further shows that refactorings performed by MANTRA are perceived to be as readable and reusable as human-written code, and in certain cases, even more favorable. These results highlight the practical advantages of MANTRA and emphasize the growing potential of LLM-based systems in advancing the automation of software refactoring tasks.