# On the Influence of Context Size and Model Choice in Retrieval-Augmented Generation Systems

arXiv: https://arxiv.org/abs/2502.14759 · HTML: https://mlsanity.com/paper/2502.14759/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation.
- Builds on 24 of its 45 references that are in the map.
- First posted on arXiv 20 Feb 2025.

## Placed in

- Retrieval-augmented generation / [RAG robustness and adaptive retrieval](https://mlsanity.com/rag/rag-robustness-and-adaptive-retrieval/)

## Builds on (most cited first, 24 of 24)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://mlsanity.com/paper/2212.10496/) (2022)
- [A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2405.06211/) (2024)
- [Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity](https://mlsanity.com/paper/2403.14403/) (2024)
- [SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization](https://mlsanity.com/paper/2111.09525/) (2021)
- [Benchmarking Retrieval-Augmented Generation for Medicine](https://mlsanity.com/paper/2402.13178/) (2024)
- [The Power of Noise: Redefining Retrieval for RAG Systems](https://mlsanity.com/paper/2401.14887/) (2024)
- [RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs](https://mlsanity.com/paper/2407.02485/) (2024)
- [A Review of Sparse Expert Models in Deep Learning](https://mlsanity.com/paper/2209.01667/) (2022)
- [A Critical Evaluation of Evaluations for Long-form Question Answering](https://mlsanity.com/paper/2305.18201/) (2023)
- [Complex Claim Verification with Evidence Retrieved in the Wild](https://mlsanity.com/paper/2305.11859/) (2023)
- [Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization](https://mlsanity.com/paper/2406.16008/) (2024)
- [LLatrieval: LLM-Verified Retrieval for Verifiable Generation](https://mlsanity.com/paper/2311.07838/) (2023)
- [DYNAMICQA: Tracing Internal Knowledge Conflicts in Language Models](https://mlsanity.com/paper/2407.17023/) (2024)
- [RetrievalSum: A Retrieval Enhanced Framework for Abstractive Summarization](https://mlsanity.com/paper/2109.07943/) (2021)
- [To Generate or to Retrieve? On the Effectiveness of Artificial Contexts for Medical Open-Domain Question Answering](https://mlsanity.com/paper/2403.01924/) (2024)
- [Does Biomedical Training Lead to Better Medical Performance?](https://mlsanity.com/paper/2404.04067/) (2024)
- [Improving Health Question Answering with Reliable and Time-Aware Evidence Retrieval](https://mlsanity.com/paper/2404.08359/) (2024)
- [MedREQAL: Examining Medical Knowledge Recall of Large Language Models via Question Answering](https://mlsanity.com/paper/2406.05845/) (2024)

## Built on by (most cited first, 5 of 5)

- [Less LLM, More Documents: Searching for Improved RAG](https://mlsanity.com/paper/2510.02657/) (2025)
- [How Retrieved Context Shapes Internal Representations in RAG](https://mlsanity.com/paper/2602.20091/) (2026)
- [MedSEBA: Synthesizing Evidence-Based Answers Grounded in Evolving Medical Literature](https://mlsanity.com/paper/2509.00414/) (2025)
- [MHA-RAG: Improving Efficiency, Accuracy, and Consistency by Encoding Exemplars as Soft Prompts](https://mlsanity.com/paper/2510.05363/) (2025)
- [Assessing the Downstream Utility of Evidence-Aware Retrieval in RAG](https://mlsanity.com/paper/2608.26379/) (2026)

## Abstract (arXiv)

Retrieval-augmented generation (RAG) has emerged as an approach to augment large language models (LLMs) by reducing their reliance on static knowledge and improving answer factuality. RAG retrieves relevant context snippets and generates an answer based on them. Despite its increasing industrial adoption, systematic exploration of RAG components is lacking, particularly regarding the ideal size of provided context, and the choice of base LLM and retrieval method. To help guide development of robust RAG systems, we evaluate various context sizes, BM25 and semantic search as retrievers, and eight base LLMs. Moving away from the usual RAG evaluation with short answers, we explore the more challenging long-form question answering in two domains, where a good answer has to utilize the entire context. Our findings indicate that final QA performance improves steadily with up to 15 snippets but stagnates or declines beyond that. Finally, we show that different general-purpose LLMs excel in the biomedical domain than the encyclopedic one, and that open-domain evidence retrieval in large corpora is challenging.