mlsanityData to 14 Oct 2026

Can Retriever-Augmented Language Models Reason? The Blame Game Between the Retriever and the Language Model

arXiv 2212.09146 ↗First posted 18 Dec 2022cs.CLcited by 32

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Augmenting pretrained language models with retrievers has shown promise in effectively solving common NLP problems, such as language modeling and question answering. In this paper, we evaluate the strengths and weaknesses of popular retriever-augmented language models, namely kNN-LM, REALM, DPR + FiD, Contriever + ATLAS, and Contriever + Flan-T5, in reasoning over retrieved statements across different tasks. Our findings indicate that the simple similarity metric employed by retrievers is insufficient for retrieving all the necessary statements for reasoning. Additionally, the language models do not exhibit strong reasoning even when provided with only the required statements. Furthermore, when combined with imperfect retrievers, the performance of the language models becomes even worse, e.g., Flan-T5's performance drops by 28.6% when retrieving 5 statements using Contriever. While larger language models improve performance, there is still a substantial room for enhancement. Our further analysis indicates that multihop retrieve-and-read is promising for large language models like GPT-3.5, but does not generalize to other language models like Flan-T5-xxl.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 23 of its 35 references that are in the map. First posted on arXiv 18 Dec 2022.

Builds on

23 of 35 references.

References · 23

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  7. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  8. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  9. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  10. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  11. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  12. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
11 more
  1. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  2. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
  3. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  4. 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 324
  5. 2019QASC: A Dataset for Question Answering via Sentence Compositioncited by 252
  6. 2020Answering Complex Open-Domain Questions with Multi-Hop Dense Retrievalcited by 162
  7. 2021Explaining Answers with Entailment Treescited by 152
  8. 2022Faithful Reasoning Using Large Language Modelscited by 130
  9. 2022Training Language Models with Memory Augmentationcited by 113
  10. 2020Answering Open-Domain Questions of Varying Reasoning Steps from Textcited by 43
  11. 2022Natural Language Deduction through Search over Statement Compositionscited by 35

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 15; 2025: 7; 2026: 2.

32 papers.

Sort:

By area:

  1. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 341
  2. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 281
  3. 2023Query Rewriting for Retrieval-Augmented Large Language Modelscited by 213
  4. 2023Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Surveycited by 92
  5. 2025ReasonIR: Training Retrievers for Reasoning Taskscited by 77
  6. 2024Reliable, Adaptable, and Attributable Language Models with Retrievalcited by 73
  7. 2023Large Language Models and Knowledge Graphs: Opportunities and Challengescited by 71
  8. 2023Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Taskscited by 64
  9. 2024Scaling Retrieval-Based Language Models with a Trillion-Token Datastorecited by 56
  10. 2023BoardgameQA: A Dataset for Natural Language Reasoning with Contradictory Informationcited by 44
  11. 2024How Easily do Irrelevant Inputs Skew the Responses of Large Language Models?cited by 44
  12. 2024Are LLMs Ready for Real-World Materials Discovery?cited by 40
20 more
  1. 2025Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidencecited by 16
  2. 2023How Well Do Large Language Models Truly Ground?cited by 14
  3. 2024RuleRAG: Rule-Guided Retrieval-Augmented Generation with Language Models for Question Answeringcited by 13
  4. 2024Dense Passage Retrieval: Is it Retrieving?cited by 12
  5. 2025ArgRAG: Explainable Retrieval Augmented Generation using Quantitative Bipolar Argumentationcited by 12
  6. 2023On Evaluating the Integration of Reasoning and Action in LLM Agents with Database Question Answeringcited by 10
  7. 2024Tell me what I need to know: Exploring LLM-based (Personalized) Abstractive Multi-Source Meeting Summarizationcited by 9
  8. 2024Adapting to Non-Stationary Environments: Multi-Armed Bandit Enhanced Retrieval-Augmented Generation on Knowledge Graphscited by 9
  9. 2025Controlled Retrieval-augmented Context Evaluation for Long-form RAGcited by 9
  10. 2024Great Memory, Shallow Reasoning: Limits of kNN-LMscited by 8
  11. 2025Frustratingly Simple Retrieval Improves Challenging, Reasoning-Intensive Benchmarkscited by 8
  12. 2024AQA: Adaptive Question Answering in a Society of LLMs via Contextual Multi-Armed Banditcited by 3
  13. 2024R^2AG: Incorporating Retrieval Information into Retrieval Augmented Generationcited by 2
  14. 2024SEGMENT+: Long Text Processing with Short-Context Language Modelscited by 2
  15. 2026RAG over Thinking Traces Can Improve Reasoning Taskscited by 2
  16. 2024Unveiling the Magic: Investigating Attention Distillation in Retrieval-augmented Generationcited by 1
  17. 2024When Life gives you LLMs, make LLM-ADE: Large Language Models with Adaptive Data Engineeringcited by 1
  18. 2025Oreo: A Plug-in Context Reconstructor to Enhance Retrieval-Augmented Generationcited by 1
  19. 2025Diverse LLMs or Diverse Question Interpretations? That is the Ensembling Questioncited by 0
  20. 2026Masking or Mitigating? Deconstructing the Impact of Query Rewriting on Retriever Biases in RAGcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Unraveling and Mitigating Retriever Inconsistencies in Retrieval-Augmented Large Language Models
  2. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Context
  3. 2026Are LLM-Based Retrievers Worth Their Cost? An Empirical Study of Efficiency, Robustness, and Reasoning Overhead
  4. 2023Reimagining Retrieval Augmented Language Models for Answering Queries
  5. 2024RAR-b: Reasoning as Retrieval Benchmark
  6. 2023BRENT: Bidirectional Retrieval Enhanced Norwegian Transformer
  7. 2025DeepRAG: Thinking to Retrieve Step by Step for Large Language Models
  8. 2026A Survey of Reasoning-Intensive Retrieval: Progress and Challenges