# Refactoring with LLMs: Bridging Human Expertise and Machine Understanding

arXiv: https://arxiv.org/abs/2510.03914 · HTML: https://mlsanity.com/paper/2510.03914/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 58 references that are in the map.
- First posted on arXiv 4 Oct 2025.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [Quantifying Memorization Across Neural Language Models](https://mlsanity.com/paper/2202.07646/) (2022)
- [CodeBLEU: a Method for Automatic Evaluation of Code Synthesis](https://mlsanity.com/paper/2009.10297/) (2020)
- [Generalizing from a Few Examples: A Survey on Few-Shot Learning](https://mlsanity.com/paper/1904.05046/) (2019)
- [Hallucination is Inevitable: An Innate Limitation of Large Language Models](https://mlsanity.com/paper/2401.11817/) (2024)
- [CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding?](https://mlsanity.com/paper/2408.10718/) (2024)
- [Refactoring Programs Using Large Language Models with Few-Shot Examples](https://mlsanity.com/paper/2311.11690/) (2023)
- [Fine-Tuning and Prompt Engineering for Large Language Models-based Code Review Automation](https://mlsanity.com/paper/2402.00905/) (2024)
- [An Empirical Study on the Code Refactoring Capability of Large Language Models](https://mlsanity.com/paper/2411.02320/) (2024)
- [An Empirical Study on the Potential of LLMs in Automated Software Refactoring](https://mlsanity.com/paper/2411.04444/) (2024)
- [CodePAD: Sequence-based Code Generation with Pushdown Automaton](https://mlsanity.com/paper/2211.00818/) (2022)
- [Knowledge-Aware Code Generation with Large Language Models](https://mlsanity.com/paper/2401.15940/) (2024)
- [Together We Go Further: LLMs and IDE Static Analysis for Extract Method Refactoring](https://mlsanity.com/paper/2401.15298/) (2024)
- [RefBERT: A Two-Stage Pre-trained Framework for Automatic Rename Refactoring](https://mlsanity.com/paper/2305.17708/) (2023)
- [COMEX: A Tool for Generating Customized Source Code Representations](https://mlsanity.com/paper/2307.04693/) (2023)
- [JExtract: An Eclipse Plug-in for Recommending Automated Extract Method Refactorings](https://mlsanity.com/paper/1506.06086/) (2015)
- [Refactoring-aware Block Tracking in Commit History](https://mlsanity.com/paper/2409.16185/) (2024)
- [An Empirical Evaluation of Impact of Refactoring On Internal and External Measures of Code Quality](https://mlsanity.com/paper/1502.03526/) (2015)
- [Refactoring Graphs: Assessing Refactoring over Time](https://mlsanity.com/paper/2003.04666/) (2020)
- [Opti Code Pro: A Heuristic Search-based Approach to Code Refactoring](https://mlsanity.com/paper/2305.07594/) (2023)
- [Evaluating the Effectiveness of Small Language Models in Detecting Refactoring Bugs](https://mlsanity.com/paper/2502.18454/) (2025)

## Built on by (most cited first, 2 of 2)

- [Lean Refactor: Multi-Objective Controllable Proof Optimization via Agentic Strategy Search](https://mlsanity.com/paper/2605.20244/) (2026)
- [SmellBench: Towards Fine-Grained Evaluation of Code Agents on Refactoring Tasks](https://mlsanity.com/paper/2606.05574/) (2026)

## Abstract (arXiv)

Code refactoring is a fundamental software engineering practice aimed at improving code quality and maintainability. Despite its importance, developers often neglect refactoring due to the significant time, effort, and resources it requires, as well as the lack of immediate functional rewards. Although several automated refactoring tools have been proposed, they remain limited in supporting a broad spectrum of refactoring types. In this study, we explore whether instruction strategies inspired by human best-practice guidelines can enhance the ability of Large Language Models (LLMs) to perform diverse refactoring tasks automatically. Leveraging the instruction-following and code comprehension capabilities of state-of-the-art LLMs (e.g., GPT-mini and DeepSeek-V3), we draw on Martin Fowler's refactoring guidelines to design multiple instruction strategies that encode motivations, procedural steps, and transformation objectives for 61 well-known refactoring types. We evaluate these strategies on benchmark examples and real-world code snippets from GitHub projects. Our results show that instruction designs grounded in Fowler's guidelines enable LLMs to successfully perform all benchmark refactoring types and preserve program semantics in real-world settings, an essential criterion for effective refactoring. Moreover, while descriptive instructions are more interpretable to humans, our results show that rule-based instructions often lead to better performance in specific scenarios. Interestingly, allowing models to focus on the overall goal of refactoring, rather than prescribing a fixed transformation type, can yield even greater improvements in code quality.