mlsanityData to 14 Oct 2026

A Systems-Level Analysis of Sensitivity, Robustness, and Stability in Retrieval-Augmented Generation

arXiv 2606.28337 ↗First posted 29 May 2026cs.IR cs.AIcited by 0

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) systems are often evaluated using final answer accuracy, even though their failures can originate from preprocessing, retrieval, context packing, or generation. This paper presents a controlled empirical study of RAG sensitivity, robustness, and stability across 56 experimental runs. We evaluate how chunk size, retrieval depth (top k), embedding-based reranking, probabilistic retrieval noise, and repeated seeded runs affect retrieval, context packing, and generation behavior. Using a fixed 500-question QA subset mapped to 20,958 unique corpus contexts, we analyze both final answer metrics and intermediate failure modes. Across these experiments, retrieval-oriented metrics improved under broader retrieval settings, while downstream exact-match and F1 scores often behaved non-monotonically. We also observe preprocessing-induced answer loss under smaller chunk sizes, progressive degradation under retrieval corruption, and higher observed variance in broader retrieval regimes. These findings suggest that RAG evaluation should include sensitivity, robustness, stability, and multi-stage failure analysis rather than relying only on final answer accuracy.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 10 of its 11 references that are in the map. First posted on arXiv 29 May 2026.

Builds on

10 of 11 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026Rethinking the Reranker: Boundary-Aware Evidence Selection for Robust Retrieval-Augmented Generation3 Feb

Other references · 9

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  5. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  6. 2017Billion-scale similarity search with GPUscited by 2,497
  7. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  8. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 265
  9. 2025DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generationcited by 15

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers
  2. 2025Optimizing Retrieval-Augmented Generation: Analysis of Hyperparameter Impact on Performance and Efficiency
  3. 2025Investigating the Robustness of Retrieval-Augmented Generation at the Query Level
  4. 2025RARE: Retrieval-Aware Robustness Evaluation for Retrieval-Augmented Generation Systems
  5. 2025Evaluating the Retrieval Robustness of Large Language Models
  6. 2026How You Ask Matters! Adaptive RAG Robustness to Query Variations
  7. 2025Out of Style: RAG's Fragility to Linguistic Variation
  8. 2025Enhancing Retrieval-Augmented Generation: A Study of Best Practices