# Repoformer: Selective Retrieval for Repository-Level Code Completion

arXiv: https://arxiv.org/abs/2403.10059 · HTML: https://mlsanity.com/paper/2403.10059/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Placed in RAG for code, one of 15 directions in Retrieval-augmented generation.
- Builds on 32 of its 43 references that are in the map.
- First posted on arXiv 15 Mar 2024.

## Placed in

- Retrieval-augmented generation / [RAG for code](https://mlsanity.com/rag/rag-for-code/)

## Builds on (most cited first, 30 of 32)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Accelerating Large Language Model Decoding with Speculative Sampling](https://mlsanity.com/paper/2302.01318/) (2023)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [In-Context Retrieval-Augmented Language Models](https://mlsanity.com/paper/2302.00083/) (2023)
- [Active Retrieval Augmented Generation](https://mlsanity.com/paper/2305.06983/) (2023)
- [CodeBLEU: a Method for Automatic Evaluation of Code Synthesis](https://mlsanity.com/paper/2009.10297/) (2020)
- [RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation](https://mlsanity.com/paper/2303.12570/) (2023)
- [UniXcoder: Unified Cross-Modal Pre-training for Code Representation](https://mlsanity.com/paper/2203.03850/) (2022)
- [The Stack: 3 TB of permissively licensed source code](https://mlsanity.com/paper/2211.15533/) (2022)
- [Efficient Training of Language Models to Fill in the Middle](https://mlsanity.com/paper/2207.14255/) (2022)
- [IntelliCode Compose: Code Generation Using Transformer](https://mlsanity.com/paper/2005.08025/) (2020)
- [CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion](https://mlsanity.com/paper/2310.11248/) (2023)
- [DocPrompting: Generating Code by Retrieving the Docs](https://mlsanity.com/paper/2207.05987/) (2022)
- [CodeGen2: Lessons for Training LLMs on Programming and Natural Languages](https://mlsanity.com/paper/2305.02309/) (2023)
- [ReACC: A Retrieval-Augmented Code Completion Framework](https://mlsanity.com/paper/2203.07722/) (2022)
- [Repository-Level Prompt Generation for Large Language Models of Code](https://mlsanity.com/paper/2206.12839/) (2022)
- [Efficient Nearest Neighbor Language Models](https://mlsanity.com/paper/2109.04212/) (2021)
- [Self-Knowledge Guided Retrieval Augmentation for Large Language Models](https://mlsanity.com/paper/2310.05002/) (2023)
- [CoCoMIC: Code Completion By Jointly Modeling In-file and Cross-file Context](https://mlsanity.com/paper/2212.10007/) (2022)
- [When Language Model Meets Private Library](https://mlsanity.com/paper/2210.17236/) (2022)
- [RepoFusion: Training Code Models to Understand Your Repository](https://mlsanity.com/paper/2306.10998/) (2023)
- [CoCoSum: Contextual Code Summarization with Multi-Relational Graph Neural Network](https://mlsanity.com/paper/2107.01933/) (2021)
- [ContraCLM: Contrastive Learning For Causal Language Model](https://mlsanity.com/paper/2210.01185/) (2022)

## Built on by (most cited first, 30 of 55)

- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph](https://mlsanity.com/paper/2410.14684/) (2024)
- [OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMs](https://mlsanity.com/paper/2504.04030/) (2025)
- [Retrieval-Augmented Code Generation: A Survey with Focus on Repository-Level Approaches](https://mlsanity.com/paper/2510.04905/) (2025)
- [What to Retrieve for Effective Retrieval-Augmented Code Generation? An Empirical Study and Beyond](https://mlsanity.com/paper/2503.20589/) (2025)
- [Perception of Knowledge Boundary for Large Language Models through Semi-open-ended Question Answering](https://mlsanity.com/paper/2405.14383/) (2024)
- [Improving Code Localization with Repository Memory](https://mlsanity.com/paper/2510.01003/) (2025)
- [An Empirical Study of Retrieval-Augmented Code Generation: Challenges and Opportunities](https://mlsanity.com/paper/2501.13742/) (2025)
- [Correctness Assessment of Code Generated by Large Language Models Using Internal Representations](https://mlsanity.com/paper/2501.12934/) (2025)
- [LLM/Agent-as-Data-Analyst: A Survey](https://mlsanity.com/paper/2509.23988/) (2025)
- [Debate as Optimization: Adaptive Conformal Prediction and Diverse Retrieval for Event Extraction](https://mlsanity.com/paper/2406.12197/) (2024)
- [RAMBO: Enhancing RAG-based Repository-Level Method Body Completion](https://mlsanity.com/paper/2409.15204/) (2024)
- [Towards an Understanding of Context Utilization in Code Intelligence](https://mlsanity.com/paper/2504.08734/) (2025)
- [Self-Routing RAG: Binding Selective Retrieval with Knowledge Verbalization](https://mlsanity.com/paper/2504.01018/) (2025)
- [CATCODER: Repository-Level Code Generation with Relevant Code and Type Context](https://mlsanity.com/paper/2406.03283/) (2024)
- [RESCUE: Retrieval Augmented Secure Code Generation](https://mlsanity.com/paper/2510.18204/) (2025)
- [AlignCoder: Aligning Retrieval with Target Intent for Repository-Level Code Completion](https://mlsanity.com/paper/2601.19697/) (2026)
- [RepoGenReflex: Enhancing Repository-Level Code Completion with Verbal Reinforcement and Retrieval-Augmented Generation](https://mlsanity.com/paper/2409.13122/) (2024)
- [Towards Realistic Project-Level Code Generation via Multi-Agent Collaboration and Semantic Architecture Modeling](https://mlsanity.com/paper/2511.03404/) (2025)
- [A Deep Dive into Retrieval-Augmented Generation for Code Completion: Experience on WeChat](https://mlsanity.com/paper/2507.18515/) (2025)
- [Enhancing Project-Specific Code Completion by Inferring Internal API Information](https://mlsanity.com/paper/2507.20888/) (2025)
- [Impact-driven Context Filtering For Cross-file Code Completion](https://mlsanity.com/paper/2508.05970/) (2025)
- [SecureReviewer: Enhancing Large Language Models for Secure Code Review through Secure-aware Fine-tuning](https://mlsanity.com/paper/2510.26457/) (2025)
- [When Parallelism Pays Off: Cohesion-Aware Task Partitioning for Multi-Agent Coding](https://mlsanity.com/paper/2606.00953/) (2026)
- [RepoReasoner: Evaluating Repository-Level Code Reasoning Ability of Long-Context Language Models](https://mlsanity.com/paper/2607.25996/) (2026)
- [Renaissance of Literate Programming in the Era of LLMs: Enhancing LLM-Based Code Generation in Large-Scale Projects](https://mlsanity.com/paper/2502.17441/) (2024)
- [Practical Code RAG at Scale: Task-Aware Retrieval Design Choices under Compute Budgets](https://mlsanity.com/paper/2510.20609/) (2025)
- [On The Importance of Reasoning for Context Retrieval in Repository-Level Code Editing](https://mlsanity.com/paper/2406.04464/) (2024)
- [TENET: One Step Toward Test-Driven Development for Repository-Level Code Generation](https://mlsanity.com/paper/2509.24148/) (2025)
- [Retrieval-augmented code completion for local projects using large language models](https://mlsanity.com/paper/2408.05026/) (2024)

## Abstract (arXiv)

Recent advances in retrieval-augmented generation (RAG) have initiated a new era in repository-level code completion. However, the invariable use of retrieval in existing methods exposes issues in both efficiency and robustness, with a large proportion of the retrieved contexts proving unhelpful or harmful to code language models (code LMs). In this paper, we propose a selective RAG framework to avoid retrieval when unnecessary. To power this framework, we design a self-supervised learning approach to enable a code LM to accurately self-evaluate whether retrieval can improve its output quality and robustly leverage the potentially noisy retrieved contexts. Using this LM as both the selective RAG policy and the generation model, our framework achieves state-of-the-art repository-level code completion performance on diverse benchmarks including RepoEval, CrossCodeEval, and CrossCodeLongEval, a new long-form code completion benchmark. Meanwhile, our analyses show that selectively retrieving brings as much as 70% inference speedup in the online serving setting without harming the performance. We further demonstrate that our framework is able to accommodate different generation models, retrievers, and programming languages. These advancements position our framework as an important step towards more accurate and efficient repository-level code completion.