# InstructCoder: Instruction Tuning Large Language Models for Code Editing

arXiv: https://arxiv.org/abs/2310.20329 · HTML: https://mlsanity.com/paper/2310.20329/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 48 references that are in the map.
- First posted on arXiv 31 Oct 2023.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks](https://mlsanity.com/paper/2204.07705/) (2022)
- [The Flan Collection: Designing Data and Methods for Effective Instruction Tuning](https://mlsanity.com/paper/2301.13688/) (2023)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [Textbooks Are All You Need II: phi-1.5 technical report](https://mlsanity.com/paper/2309.05463/) (2023)
- [InCoder: A Generative Model for Code Infilling and Synthesis](https://mlsanity.com/paper/2204.05999/) (2022)
- [Is ChatGPT a Good NLG Evaluator? A Preliminary Study](https://mlsanity.com/paper/2303.04048/) (2023)
- [Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor](https://mlsanity.com/paper/2212.09689/) (2022)
- [ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning](https://mlsanity.com/paper/2111.10952/) (2021)
- [Learning Performance-Improving Code Edits](https://mlsanity.com/paper/2302.07867/) (2023)
- [CODIT: Code Editing with Tree-Based Neural Models](https://mlsanity.com/paper/1810.00314/) (2018)
- [Learning to Represent Edits](https://mlsanity.com/paper/1810.13337/) (2018)
- [Learning to Fix Build Errors with Graph2Diff Neural Networks](https://mlsanity.com/paper/1911.01205/) (2019)
- [Patching as Translation: the Data and the Metaphor](https://mlsanity.com/paper/2008.10707/) (2020)
- [Coeditor: Leveraging Contextual Changes for Multi-round Code Auto-editing](https://mlsanity.com/paper/2305.18584/) (2023)

## Built on by (most cited first, 29 of 29)

- [Continual Learning of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2404.16789/) (2024)
- [MMCode: Benchmarking Multimodal Large Language Models for Code Generation with Visually Rich Programming Problems](https://mlsanity.com/paper/2404.09486/) (2024)
- [Securing Large Language Models: Addressing Bias, Misinformation, and Prompt Attacks](https://mlsanity.com/paper/2409.08087/) (2024)
- [Genius: A Generalizable and Purely Unsupervised Self-Training Framework For Advanced Reasoning](https://mlsanity.com/paper/2504.08672/) (2025)
- [WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation](https://mlsanity.com/paper/2508.16763/) (2025)
- [Speculative Decoding: Performance or Illusion?](https://mlsanity.com/paper/2601.11580/) (2025)
- [COAST: Enhancing the Code Debugging Ability of LLMs through Communicative Agent Based Data Synthesis](https://mlsanity.com/paper/2408.05006/) (2024)
- [Risk Assessment Framework for Code LLMs via Leveraging Internal States](https://mlsanity.com/paper/2504.14640/) (2025)
- [Training Language Models on Synthetic Edit Sequences Improves Code Synthesis](https://mlsanity.com/paper/2410.02749/) (2024)
- [CodeLutra: Boosting LLM Code Generation via Preference-Guided Refinement](https://mlsanity.com/paper/2411.05199/) (2024)
- [Efficient MoE Serving in the Memory-Bound Regime: Balance Activated Experts, Not Tokens](https://mlsanity.com/paper/2512.09277/) (2025)
- [Evaluating the Effectiveness of LLMs in Fixing Maintainability Issues in Real-World Projects](https://mlsanity.com/paper/2502.02368/) (2025)
- [EditLord: Learning Code Transformation Rules for Code Editing](https://mlsanity.com/paper/2504.15284/) (2025)
- [Efficiency vs. Alignment: Investigating Safety and Fairness Risks in Parameter-Efficient Fine-Tuning of LLMs](https://mlsanity.com/paper/2511.00382/) (2025)
- [NES: An Instruction-Free, Low-Latency Next Edit Suggestion Framework Powered by Learned Historical Editing Trajectories](https://mlsanity.com/paper/2508.02473/) (2025)
- [To Diff or Not to Diff? Structure-Aware and Adaptive Output Formats for Efficient LLM-based Code Editing](https://mlsanity.com/paper/2604.27296/) (2026)
- [AMR-Evol: Adaptive Modular Response Evolution Elicits Better Knowledge Distillation for Large Language Models in Code Generation](https://mlsanity.com/paper/2410.00558/) (2024)
- [Enhancing the Non-Functional Quality Compliance of LLM-Generated Code through Quality-Aware Preference Learning](https://mlsanity.com/paper/2503.09020/) (2025)
- [Automatic Qiskit Code Refactoring Using Large Language Models](https://mlsanity.com/paper/2506.14535/) (2025)
- [Symphony-MoE: Harmonizing Disparate Pre-trained Models into a Coherent Mixture-of-Experts](https://mlsanity.com/paper/2509.18542/) (2025)
- [Breaking Memorization Barriers in LLM Code Fine-Tuning via Information Bottleneck for Improved Generalization](https://mlsanity.com/paper/2510.16022/) (2025)
- [Measurement of Generative AI Workload Power Profiles for Whole-Facility Data Center Infrastructure Planning](https://mlsanity.com/paper/2604.07345/) (2026)
- [Learning from 53.6K Real-World Developer Edits of AI-Generated Code](https://mlsanity.com/paper/2607.25130/) (2026)
- [EdiTikZ: Scientific Figure Editing from Revision Trajectories](https://mlsanity.com/paper/2609.01409/) (2026)
- [TACOS: Open Tagging and Comparative Scoring for Instruction Fine-Tuning Data Selection](https://mlsanity.com/paper/2507.03673/) (2025)
- [Reward-Free Code Alignment from Pretrained or Fine-Tuned LLM: Unpacking the Trade-offs for Code Generation](https://mlsanity.com/paper/2606.28998/) (2026)
- [Diff-Based Code Corruption using LLMs for Large-Scale Bugfix Benchmarking](https://mlsanity.com/paper/2606.29088/) (2026)
- [Quantize with Confidence? An Empirical Study of Quantization for Code Generation](https://mlsanity.com/paper/2607.14181/) (2026)
- [Diffs vs. Whole Files: An Empirical Comparison of Iterative Edit-Based and Direct Generation for Flutter/Dart Code Models](https://mlsanity.com/paper/2609.05779/) (2026)

## Abstract (arXiv)

Code editing encompasses a variety of pragmatic tasks that developers deal with daily. Despite its relevance and practical usefulness, automatic code editing remains an underexplored area in the evolution of deep learning models, partly due to data scarcity. In this work, we explore the use of Large Language Models (LLMs) to edit code based on user instructions. Evaluated on a novel human-written execution-based benchmark dubbed EditEval, we found current models often struggle to fulfill the instructions. In light of this, we contribute InstructCoder, the first instruction-tuning dataset designed to adapt LLMs for general-purpose code editing, containing high-diversity code-editing tasks such as comment insertion, code optimization, and code refactoring. It consists of over 114,000 instruction-input-output triplets and covers multiple distinct code editing scenarios. The collection process starts with filtered commit data sourced from GitHub Python repositories as seeds. Subsequently, the dataset is systematically expanded through an iterative process, where both seed and generated tasks are used to prompt ChatGPT for more data. Our findings reveal that open-source LLMs fine-tuned on InstructCoder can significantly enhance the accuracy of code edits, exhibiting superior code-editing performance matching advanced proprietary LLMs. The datasets and the source code are publicly available at https://github.com/qishenghu/CodeInstruct.