mlsanityData to 7 Oct 2026

Understanding Retrieval Augmentation for Long-Form Question Answering

arXiv 2310.12150 ↗First posted 18 Oct 2023cs.CLcited by 45

Not on the map. Why?

Abstract (arXiv)

How retrieved documents are used in language models (LMs) for long-form generation task is understudied. We present two controlled studies on retrieval-augmented LM for long-form question answering (LFQA): one fixing the LM and varying evidence documents and the other fixing evidence documents and varying the LMs. We study various attributes of generated answers (e.g., fluency, length, variance), with an emphasis on the attribution of generated answers to in-context evidence documents. We collect a dataset (SALAD) containing human annotations of sentence-level answer attribution in LFQA and evaluate existing methods for automatically judging attribution. We find that while LMs can leverage relevant in-context documents, the generated answer is only partially attributable towards the documents, especially for LMs trained without retrieval augmentation. Together, our analysis reveals how retrieval augmentation impacts long knowledge-rich text generation and provide directions for future work.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Builds on 42 of its 49 references that are in the map. First posted on arXiv 18 Oct 2023.

Builds on

42 of 49 references.

References · 42

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  5. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  7. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  8. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  9. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  10. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  11. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  12. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
30 more
  1. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  2. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  3. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  4. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  5. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  6. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  7. 2019ELI5: Long Form Question Answeringcited by 567
  8. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
  9. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 402
  10. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 349
  11. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  12. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  13. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
  14. 2021QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarizationcited by 233
  15. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 231
  16. 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 222
  17. 2019MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehensioncited by 183
  18. 2021Hurdles to Progress in Long-form Question Answeringcited by 166
  19. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 142
  20. 2022Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluationcited by 141
  21. 2023WiCE: Real-World Entailment for Claims in Wikipediacited by 118
  22. 2023A Critical Evaluation of Evaluations for Long-form Question Answeringcited by 111
  23. 2022Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Modelscited by 108
  24. 2023ExpertQA: Expert-Curated Questions and Attributed Answerscited by 96
  25. 2023Context-faithful Prompting for Large Language Modelscited by 76
  26. 2021DocNLI: A Large-scale Dataset for Document-level Natural Language Inferencecited by 75
  27. 2022RankGen: Improving Text Generation with Large Ranking Modelscited by 72
  28. 2023Automatic Evaluation of Attribution by Large Language Modelscited by 70
  29. 2022Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusterscited by 44
  30. 2023KNN-LM Does Not Improve Open-ended Text Generationcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 24; 2025: 14; 2026: 4.

45 papers.

Sort:

By area:

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  2. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  3. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 251
  4. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 170
  5. 2024RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generationcited by 97
  6. 2024Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systemscited by 87
  7. 2023A Survey of Large Language Models Attributioncited by 71
  8. 2024RA-ISF: Learning to Answer and Understand from Retrieval Augmentation via Iterative Self-Feedbackcited by 46
  9. 2024Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Modelscited by 43
  10. 2025DeepRAG: Thinking to Retrieve Step by Step for Large Language Modelscited by 37
  11. 2024AttributionBench: How Hard is Automatic Attribution Evaluation?cited by 26
  12. 2024DnDScore: Decontextualization and Decomposition for Factuality Verification in Long-Form Text Generationcited by 22
33 more
  1. 2024How well do LLMs cite relevant medical references? An evaluation framework and analysescited by 20
  2. 2024Evaluation of RAG Metrics for Question Answering in the Telecom Domaincited by 19
  3. 2024From Single to Multi: How LLMs Hallucinate in Multi-Document Summarizationcited by 17
  4. 2024ALiiCE: Evaluating Positional Fine-grained Citation Generationcited by 15
  5. 2024Losing Visual Needles in Image Haystacks: Vision Language Models are Easily Distracted in Short and Long Contextscited by 14
  6. 2026Relevant Is Not Warranted: Evidence-Force Calibration for Cited RAGcited by 14
  7. 2025Verifying the Verifiers: Unveiling Pitfalls and Potentials in Fact Verifierscited by 12
  8. 2024Large Language Models for Social Networks: Applications, Challenges, and Solutionscited by 11
  9. 2024Retrieval-Enhanced Machine Learning: Synthesis and Opportunitiescited by 11
  10. 2025LeTS: Learning to Think-and-Search via Process-and-Outcome Reward Hybridizationcited by 10
  11. 2024Do Automatic Factuality Metrics Measure Factuality? A Critical Evaluationcited by 9
  12. 2025CoCoA: Confidence and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in Large Language Modelscited by 9
  13. 2024Observations on Building RAG Systems for Technical Documentscited by 8
  14. 2025Verify with Caution: The Pitfalls of Relying on Imperfect Factuality Metricscited by 8
  15. 2024Groundedness in Retrieval-augmented Long-form Generation: An Empirical Studycited by 7
  16. 2025Cite Before You Speak: Enhancing Context-Response Grounding in E-commerce Conversational LLM-Agentscited by 7
  17. 2025Linguistic Nepotism: Trading-off Quality for Language Preference in Multilingual RAGcited by 7
  18. 2024Structure-aware Domain Knowledge Injection for Large Language Modelscited by 6
  19. 2024Experimenting with Legal AI Solutions: The Case of Question-Answering for Access to Justicecited by 5
  20. 2025Reasoning-CV: Fine-tuning Powerful Reasoning LLMs for Knowledge-Assisted Claim Verificationcited by 5
  21. 2025CLATTER: Comprehensive Entailment Reasoning for Hallucination Detectioncited by 5
  22. 2025Document Intelligence in the Era of Large Language Models: A Surveycited by 5
  23. 2024LFQA-E: Carefully Benchmarking Long-form QA Evaluationcited by 3
  24. 2025A Systematic Framework for Enterprise Knowledge Retrieval: Leveraging LLM-Generated Metadata to Enhance RAG Systemscited by 3
  25. 2025FaithLens: Detecting and Explaining Faithfulness Hallucinationcited by 3
  26. 2024Auto-GDA: Automatic Domain Adaptation for Efficient Grounding Verification in Retrieval-Augmented Generationcited by 2
  27. 2025Reinforced Informativeness Optimization for Long-Form Retrieval-Augmented Generationcited by 2
  28. 2025HalluGuard: Evidence-Grounded Small Reasoning Models to Mitigate Hallucinations in Retrieval-Augmented Generationcited by 2
  29. 2024Science Checker Reloaded: A Bidirectional Paradigm for Transparency and Logical Reasoningcited by 0
  30. 2024Zero-shot Factual Consistency Evaluation Across Domainscited by 0
  31. 2026ThinknCheck: Grounded Claim Verification with Compact, Reasoning-Driven, and Interpretable Modelscited by 0
  32. 2026Teaching Language Models to Check Grounded Claim Factuality with Human Test-Taking Strategiescited by 0
  33. 2026Averaging Bias: Human Faithfulness Annotations are not Locally Faithfulcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA
  2. 2024LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering
  3. 2026Predicting the Benefit of Retrieval Augmentation in Open-Domain Question Answering
  4. 2026How Do LLMs Cite? A Mechanistic Interpretation of Attribution in Retrieval-Augmented Generation
  5. 2024RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answering
  6. 2026EnSI-RAG: Entity-Structure-Indexed Retrieval-Augmented Generation for Long-Document Question Answering
  7. 2024Investigating Context-Faithfulness in Large Language Models: The Roles of Memory Strength and Evidence Style
  8. 2025On the Influence of Context Size and Model Choice in Retrieval-Augmented Generation Systems