mlsanityData to 14 Oct 2026

Evaluating the Effectiveness and Efficiency of Demonstration Retrievers in RAG for Coding Tasks

arXiv 2410.09662 ↗First posted 12 Oct 2024cs.SEcited by 2

On the map: Retrieval-augmented generation → RAG for code

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by integrating external knowledge bases, achieving state-of-the-art results in various coding tasks. The core of RAG is retrieving demonstration examples, which is essential to balance effectiveness (generation quality) and efficiency (retrieval time) for optimal performance. However, the high-dimensional nature of code representations and large knowledge bases often create efficiency bottlenecks, which are overlooked in previous research. This paper systematically evaluates the efficiency-effectiveness trade-off of retrievers across three coding tasks: Program Synthesis, Commit Message Generation, and Assertion Generation. We examined six retrievers: two sparse (BM25 and BM25L) and four dense retrievers, including one exhaustive dense retriever (SBERT's Semantic Search) and three approximate dense retrievers (ANNOY, LSH, and HNSW). Our findings show that while BM25 excels in effectiveness, it suffers in efficiency as the knowledge base grows beyond 1000 entries. In large-scale retrieval, efficiency differences become more pronounced, with approximate dense retrievers offering the greatest gains. For instance, in Commit Generation task, HNSW achieves a 44x speed up, while only with a 1.74% drop in RougeL compared with BM25. Our results also show that increasing the number of demonstrations in the prompt doesn't always improve the effectiveness and can increase latency and lead to incorrect outputs. Our findings provide valuable insights for practitioners aiming to build efficient and effective RAG systems for coding tasks.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RAG for code, one of 15 directions in Retrieval-augmented generation. Builds on 24 of its 39 references that are in the map. First posted on arXiv 12 Oct 2024.

Builds on

24 of 39 references.

References · 24

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  3. 2017Billion-scale similarity search with GPUscited by 2,497
  4. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  5. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  6. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
  7. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  8. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  9. 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 479
  10. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 453
  11. 2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 361
  12. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 348
12 more
  1. 2020Pretrained Transformers for Text Ranking: BERT and Beyondcited by 267
  2. 2018Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflowcited by 165
  3. 2022DocPrompting: Generating Code by Retrieving the Docscited by 153
  4. 2018ANN-Benchmarks: A Benchmarking Tool for Approximate Nearest Neighbor Algorithmscited by 130
  5. 2024Evaluating Retrieval Quality in Retrieval-Augmented Generationcited by 90
  6. 2024Blended RAG: Improving RAG (Retriever-Augmented Generation) Accuracy with Semantic Search and Hybrid Query-Based Retrieverscited by 84
  7. 2021A Comprehensive Survey and Experimental Comparison of Graph-Based Approximate Nearest Neighbor Searchcited by 82
  8. 2023How Many Demonstrations Do You Need for In-context Learning?cited by 52
  9. 2023What Makes Good In-context Demonstrations for Code Intelligence Tasks with LLMs?cited by 47
  10. 2020On Learning Meaningful Assert Statements for Unit Test Casescited by 42
  11. 2021A Survey on Locality Sensitive Hashing Algorithms and their Applicationscited by 40
  12. 2024The Chronicles of RAG: The Retriever, the Chunk and the Generatorcited by 34

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026NasZip: Software and Hardware Co-Design to Accelerate Approximate Nearest Neighbor Search with DIMM-Based Near-Data Processingcited by 0
  2. 2026Retrieval-Oriented Code Representations in Agentic Bug Localizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning
  2. 2024CodeRAG-Bench: Can Retrieval Augment Code Generation?
  3. 2025A Deep Dive into Retrieval-Augmented Generation for Code Completion: Experience on WeChat
  4. 2024Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systems
  5. 2024Toward Optimal Search and Retrieval for RAG
  6. 2024XRAG: eXamining the Core -- Benchmarking Foundational Components in Advanced Retrieval-Augmented Generation
  7. 2025Less Is More: Elevating RAG via Performance-Driven Context Compression
  8. 2025What to Retrieve for Effective Retrieval-Augmented Code Generation? An Empirical Study and Beyond