mlsanityData to 7 Oct 2026

Reimagining Retrieval Augmented Language Models for Answering Queries

arXiv 2306.01061 ↗First posted 1 Jun 2023cs.CL cs.DBcited by 2

On the map: Retrieval-augmented generation → Generation and context integration

Abstract (arXiv)

We present a reality check on large language models and inspect the promise of retrieval augmented language models in comparison. Such language models are semi-parametric, where models integrate model parameters and knowledge from external data sources to make their predictions, as opposed to the parametric nature of vanilla large language models. We give initial experimental findings that semi-parametric architectures can be enhanced with views, a query analyzer/planner, and provenance to make a significantly more powerful system for question answering in terms of accuracy and efficiency, and potentially for other NLP tasks

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 50 of its 95 references that are in the map. First posted on arXiv 1 Jun 2023.

Builds on

50 of 95 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Models5 Aug
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks22 May
  3. 2020REALM: Retrieval-Augmented Language Model Pre-Training10 Feb

Other references · 47

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2020Scaling Laws for Neural Language Modelscited by 6,411
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  9. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  10. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  11. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  12. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
35 more
  1. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  2. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  3. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  4. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  5. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 565
  6. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  7. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
  8. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
  9. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 402
  10. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 385
  11. 2022RealTime QA: What's the Answer Right Now?cited by 239
  12. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 233
  13. 2020Selective Question Answering under Domain Shiftcited by 228
  14. 2022Binding Language Models in Symbolic Languagescited by 224
  15. 2022MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Textcited by 216
  16. 2019CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databasescited by 180
  17. 2020Answering Complex Open-Domain Questions with Multi-Hop Dense Retrievalcited by 160
  18. 2018On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detectioncited by 159
  19. 2016Deep Image Retrieval: Learning global representations for image searchcited by 147
  20. 2020Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functionscited by 144
  21. 2022CM3: A Causal Masked Multimodal Model of the Internetcited by 142
  22. 2022KNN-Diffusion: Image Generation via Large-Scale Retrievalcited by 135
  23. 2020Break It Down: A Question Understanding Benchmarkcited by 131
  24. 2022Retrieval-Augmented Multimodal Language Modelingcited by 125
  25. 2022RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQLcited by 85
  26. 2021Anticipating Safety Issues in E2E Conversational AI: Framework and Toolingcited by 77
  27. 2021Open Domain Question Answering over Tables via Dense Retrievalcited by 74
  28. 2020UniK-QA: Unified Representations of Structured and Unstructured Knowledge for Open-Domain Question Answeringcited by 61
  29. 2020MOCHA: A Dataset for Training and Evaluating Generative Reading Comprehension Metricscited by 42
  30. 2021Evidentiality-guided Generation for Knowledge-Intensive NLP Taskscited by 40
  31. 2020To Test Machine Comprehension, Start by Defining Comprehensioncited by 31
  32. 2021Capturing Row and Column Semantics in Transformer Based Question Answering over Tablescited by 31
  33. 2021Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrievalcited by 25
  34. 2021What's in a Name? Answer Equivalence For Open-Domain Question Answeringcited by 21
  35. 2022Reasoning over Public and Private Data in Retrieval-Based Systemscited by 21

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1.

2 papers.

  1. 2023Large Language Models and Knowledge Graphs: Opportunities and Challengescited by 71
  2. 2024LaPuda: LLM-Enabled Policy-Based Query Optimizer for Multi-modal Datacited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Reliable, Adaptable, and Attributable Language Models with Retrieval
  2. 2023BRENT: Bidirectional Retrieval Enhanced Norwegian Transformer
  3. 2022Can Retriever-Augmented Language Models Reason? The Blame Game Between the Retriever and the Language Model
  4. 2024Query Optimization for Parametric Knowledge Refinement in Retrieval-Augmented Large Language Models
  5. 2024From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queries
  6. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Context
  7. 2024Unraveling and Mitigating Retriever Inconsistencies in Retrieval-Augmented Large Language Models
  8. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity