# Out of Context: How important is Local Context in Neural Program Repair?

arXiv: https://arxiv.org/abs/2312.04986 · HTML: https://mlsanity.com/paper/2312.04986/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 52 references that are in the map.
- First posted on arXiv 8 Dec 2023.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://mlsanity.com/paper/2108.12409/) (2021)
- [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021)
- [SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repair](https://mlsanity.com/paper/1901.01808/) (2018)
- [CURE: Code-Aware Neural Machine Translation for Automatic Program Repair](https://mlsanity.com/paper/2103.00073/) (2021)
- [Impact of Code Language Models on Automated Program Repair](https://mlsanity.com/paper/2302.05020/) (2023)
- [On the Robustness of Code Generation Techniques: An Empirical Study on GitHub Copilot](https://mlsanity.com/paper/2302.00438/) (2023)
- [RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repair](https://mlsanity.com/paper/2309.06057/) (2023)
- [How Often Do Single-Statement Bugs Occur? The ManySStuBs4J Dataset](https://mlsanity.com/paper/1905.13334/) (2019)
- [ARJA: Automated Repair of Java Programs via Multi-Objective Genetic Programming](https://mlsanity.com/paper/1712.07804/) (2017)
- [Elixir: Effective object-oriented program repair](https://mlsanity.com/paper/2112.10915/) (2021)
- [TSSB-3M: Mining single statement bugs at massive scale](https://mlsanity.com/paper/2201.12046/) (2022)
- [Megadiff: A Dataset of 600k Java Source Code Changes Categorized by Diff Size](https://mlsanity.com/paper/2108.04631/) (2021)
- [JEMMA: An Extensible Java Dataset for ML4Code Applications](https://mlsanity.com/paper/2212.09132/) (2022)

## Built on by (most cited first, 5 of 5)

- [Towards an Understanding of Context Utilization in Code Intelligence](https://mlsanity.com/paper/2504.08734/) (2025)
- [REDO: Execution-Free Runtime Error Detection for COding Agents](https://mlsanity.com/paper/2410.09117/) (2024)
- [Repairing Regex Vulnerabilities via Localization-Guided Instructions](https://mlsanity.com/paper/2510.09037/) (2025)
- [Human-Aligned Enhancement of Programming Answers with LLMs Guided by User Feedback](https://mlsanity.com/paper/2601.17604/) (2026)
- [On the Impact of Code Comments for Automated Bug-Fixing: An Empirical Study](https://mlsanity.com/paper/2601.23059/) (2026)

## Abstract (arXiv)

Deep learning source code models have been applied very successfully to the problem of automated program repair. One of the standing issues is the small input window of current models which often cannot fully fit the context code required for a bug fix (e.g., method or class declarations of a project). Instead, input is often restricted to the local context, that is, the lines below and above the bug location. In this work we study the importance of this local context on repair success: how much local context is needed?; is context before or after the bug location more important? how is local context tied to the bug type? To answer these questions we train and evaluate Transformer models in many different local context configurations on three datasets and two programming languages. Our results indicate that overall repair success increases with the size of the local context (albeit not for all bug types) and confirm the common practice that roughly 50-60% of the input window should be used for context leading the bug. Our results are not only relevant for researchers working on Transformer-based APR tools but also for benchmark and dataset creators who must decide what and how much context to include in their datasets.