mlsanityData to 14 Oct 2026

Unraveling and Mitigating Retriever Inconsistencies in Retrieval-Augmented Large Language Models

arXiv 2405.20680 ↗First posted 31 May 2024cs.AI cs.CLcited by 2

On the map: Retrieval-augmented generation → Generation and context integration

Abstract (arXiv)

Although Retrieval-Augmented Large Language Models (RALMs) demonstrate their superiority in terms of factuality, they do not consistently outperform the original retrieval-free Language Models (LMs). Our experiments reveal that this example-level performance inconsistency exists not only between retrieval-augmented and retrieval-free LM but also among different retrievers. To understand this phenomenon, we investigate the degeneration behavior of RALMs and theoretically decompose it into four categories. Further analysis based on our decomposition reveals that the innate difference in knowledge sources and the unpredictable degeneration of the reader model contribute most to the inconsistency. Drawing from our analysis, we introduce Ensemble of Retrievers (EoR), a trainable framework that can adaptively retrieve from different knowledge sources and effectively decrease unpredictable reader errors. Our experiments on Open Domain Question Answering show that EoR substantially improves performance over the RALM with a single retriever by considerably reducing inconsistent behaviors.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 30 of its 40 references that are in the map. First posted on arXiv 31 May 2024.

Builds on

30 of 40 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation6 Oct
  2. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Context2 Oct
  3. 2023In-Context Retrieval-Augmented Language Models31 Jan
  4. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories20 Dec
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks22 May

Other references · 25

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  7. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  8. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  9. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  10. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  11. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  12. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
13 more
  1. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  2. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  3. 2023Enabling Large Language Models to Generate Text with Citationscited by 595
  4. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
  5. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 318
  6. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 298
  7. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 296
  8. 2022Teaching language models to support answers with verified quotescited by 292
  9. 2022Large Language Models with Controllable Working Memorycited by 169
  10. 2023Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentationcited by 147
  11. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 144
  12. 2023Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarizationcited by 122
  13. 2023WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferencescited by 100

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Surveycited by 20
  2. 2024E2LLM: Encoder Elongated Large Language Models for Long-Context Understanding and Reasoningcited by 13

Similar, not cited

Close in wording, no citation either way

  1. 2024Assessing "Implicit" Retrieval Robustness of Large Language Models
  2. 2022Can Retriever-Augmented Language Models Reason? The Blame Game Between the Retriever and the Language Model
  3. 2024Reliable, Adaptable, and Attributable Language Models with Retrieval
  4. 2024RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing
  5. 2023BRENT: Bidirectional Retrieval Enhanced Norwegian Transformer
  6. 2025Do Retrieval-Augmented Language Models Adapt to Varying User Needs?
  7. 2023RaLLe: A Framework for Developing and Evaluating Retrieval-Augmented Large Language Models
  8. 2023Reimagining Retrieval Augmented Language Models for Answering Queries