# MORepair: Teaching LLMs to Repair Code via Multi-Objective Fine-tuning

arXiv: https://arxiv.org/abs/2404.12636 · HTML: https://mlsanity.com/paper/2404.12636/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Apr 2024.

## Built on by (most cited first, 13 of 13)

- [From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future](https://mlsanity.com/paper/2408.02479/) (2024)
- [When LLMs Meet Cybersecurity: A Systematic Literature Review](https://mlsanity.com/paper/2405.03644/) (2024)
- [A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic System](https://mlsanity.com/paper/2510.09721/) (2025)
- [From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence](https://mlsanity.com/paper/2511.18538/) (2025)
- [A Taxonomy of Prompt Defects in LLM Systems](https://mlsanity.com/paper/2509.14404/) (2025)
- ["My productivity is boosted, but ..." Demystifying Users' Perception on AI Coding Assistants](https://mlsanity.com/paper/2508.12285/) (2025)
- [Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanism](https://mlsanity.com/paper/2510.13170/) (2025)
- [FailureMem: A Failure-Aware Multimodal Framework for Autonomous Software Repair](https://mlsanity.com/paper/2603.17826/) (2026)
- [SelfHeal: Empirical Fix Pattern Analysis and Bug Repair in LLM Agents](https://mlsanity.com/paper/2604.17699/) (2026)
- [BoostAPR: Boosting Automated Program Repair via Execution-Grounded Reinforcement Learning with Dual Reward Models](https://mlsanity.com/paper/2605.09134/) (2026)
- [PassNet: Scaling Large Language Models for Graph Compiler Pass Generation](https://mlsanity.com/paper/2605.29357/) (2026)
- [When the Code Autopilot Breaks: Why LLMs Falter in Embedded Machine Learning](https://mlsanity.com/paper/2509.10946/) (2025)
- [Knowledge-Enhanced Agentic Vulnerability Repair](https://mlsanity.com/paper/2607.00820/) (2026)

## Abstract (arXiv)

Within the realm of software engineering, specialized tasks on code, such as program repair, present unique challenges, necessitating fine-tuning Large language models~(LLMs) to unlock state-of-the-art performance. Fine-tuning approaches proposed in the literature for LLMs on program repair tasks generally overlook the need to reason about the logic behind code changes, beyond syntactic patterns in the data. High-performing fine-tuning experiments also usually come at very high computational costs. With MORepair, we propose a novel perspective on the learning focus of LLM fine-tuning for program repair: we not only adapt the LLM parameters to the syntactic nuances of the task of code transformation (objective 1), but we also specifically fine-tune the LLM with respect to the logical reason behind the code change in the training data (objective 2). Such a multi-objective fine-tuning will instruct LLMs to generate high-quality patches. We apply MORepair to fine-tune four open-source LLMs with different sizes and architectures. Experimental results on function-level and repository-level repair benchmarks show that the implemented fine-tuning effectively boosts LLM repair performance by 11.4% to 56.0%. We further show that our fine-tuning strategy yields superior performance compared to the state-of-the-art approaches, including standard fine-tuning, Fine-tune-CoT, and RepairLLaMA.