mlsanityData to 7 Oct 2026

ASQA: Factoid Questions Meet Long-Form Answers

arXiv 2204.06092 ↗First posted 12 Apr 2022cs.CLcited by 231

Not on the map. Why?

Cited by 88 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

An abundance of datasets and availability of reliable evaluation metrics have resulted in strong progress in factoid question answering (QA). This progress, however, does not easily transfer to the task of long-form QA, where the goal is to answer questions that require in-depth explanations. The hurdles include (i) a lack of high-quality data, and (ii) the absence of a well-defined notion of the answer's quality. In this work, we address these problems by (i) releasing a novel dataset and a task that we call ASQA (Answer Summaries for Questions which are Ambiguous); and (ii) proposing a reliable metric for measuring performance on ASQA. Our task focuses on factoid questions that are ambiguous, that is, have different correct answers depending on interpretation. Answers to ambiguous questions should synthesize factual information from multiple sources into a long-form summary that resolves the ambiguity. In contrast to existing long-form QA tasks (such as ELI5), ASQA admits a clear notion of correctness: a user faced with a good summary should be able to answer different interpretations of the original ambiguous question. We use this notion of correctness to define an automated metric of performance for ASQA. Our analysis demonstrates an agreement between this metric and human judgments, and reveals a considerable gap between human performance and strong baselines.

Key facts. Cited by 231 papers in the map (data through 7 Oct 2026). Builds on 34 of its 47 references that are in the map. First posted on arXiv 12 Apr 2022.

Builds on

34 of 47 references.

References · 34

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  8. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  9. 2020Learning to summarize from human feedbackcited by 2,562
  10. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  11. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  12. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
22 more
  1. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  2. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  3. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  4. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  5. 2018CoQA: A Conversational Question Answering Challengecited by 674
  6. 2017The NarrativeQA Reading Comprehension Challengecited by 590
  7. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  8. 2019ELI5: Long Form Question Answeringcited by 567
  9. 2019Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Modelcited by 431
  10. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 385
  11. 2018QuAC : Question Answering in Contextcited by 379
  12. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
  13. 2021QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarizationcited by 363
  14. 2018Generating Wikipedia by Summarizing Long Sequencescited by 333
  15. 2016NewsQA: A Machine Comprehension Datasetcited by 330
  16. 2020FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarizationcited by 292
  17. 2021Hurdles to Progress in Long-form Question Answeringcited by 166
  18. 2021Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answeringcited by 131
  19. 2021Joint Passage Ranking for Diverse Multi-Answer Retrievalcited by 43
  20. 2017Diversity driven Attention Model for Query-based Abstractive Summarizationcited by 42
  21. 2018ComQA: A Community-sourced Dataset for Complex Factoid Question Answering with Paraphrase Clusterscited by 29
  22. 2021ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answerscited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 24; 2024: 83; 2025: 74; 2026: 47.

231 papers.

Sort:

By area:

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  2. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  3. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  4. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  5. 2023Active Retrieval Augmented Generationcited by 506
  6. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 399
  7. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  8. 2023FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Setscited by 198
  9. 2024FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Researchcited by 165
  10. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generationcited by 144
  11. 2024HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughlycited by 119
  12. 2022Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Modelscited by 108
38 more
  1. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
  2. 2023ExpertQA: Expert-Curated Questions and Attributed Answerscited by 96
  3. 2024Searching for Best Practices in Retrieval-Augmented Generationcited by 83
  4. 2024LUQ: Long-text Uncertainty Quantification for LLMscited by 79
  5. 2024Speculative RAG: Enhancing Retrieval Augmented Generation through Draftingcited by 76
  6. 2023TIGERScore: Towards Building Explainable Metric for All Text Generation Taskscited by 69
  7. 2024SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMscited by 67
  8. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 65
  9. 2023Selectively Answering Ambiguous Questionscited by 63
  10. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  11. 2023PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documentscited by 61
  12. 2024Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Modelscited by 57
  13. 2024Aligning Language Models to Explicitly Handle Ambiguitycited by 55
  14. 2024Attribute First, then Generate: Locally-attributable Grounded Text Generationcited by 54
  15. 2023LLatrieval: LLM-Verified Retrieval for Verifiable Generationcited by 52
  16. 2025MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systemscited by 50
  17. 2024LongGenBench: Benchmarking Long-Form Generation in Long Context LLMscited by 47
  18. 2025A Survey of Large Language Model Agents for Question Answeringcited by 46
  19. 2023Understanding Retrieval Augmentation for Long-Form Question Answeringcited by 45
  20. 2022Conditional Generation with a Question-Answering Blueprintcited by 44
  21. 2023Effective Large Language Model Adaptation for Improved Grounding and Citation Generationcited by 44
  22. 2024RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answeringcited by 43
  23. 2024MAIN-RAG: Multi-Agent Filtering Retrieval-Augmented Generationcited by 43
  24. 2025An Empirical Study on Reinforcement Learning for Reasoning-Search Interleaved LLM Agentscited by 43
  25. 2023Tree of Clarifications: Answering Ambiguous Questions with Retrieval-Augmented Large Language Modelscited by 42
  26. 2024HelloBench: Evaluating Long Text Generation Capabilities of Large Language Modelscited by 42
  27. 2023LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedbackcited by 39
  28. 2024Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMscited by 34
  29. 2024Measuring and Enhancing Trustworthiness of LLMs in RAG through Grounded Attributions and Learning to Refusecited by 33
  30. 2024Long2RAG: Evaluating Long-Context & Long-Form Retrieval-Augmented Generation with Key Point Recallcited by 32
  31. 2024Genie: Achieving Human Parity in Content-Grounded Datasets Generationcited by 31
  32. 2024Calibrating Long-form Generations from Large Language Modelscited by 31
  33. 2024Ragnar"ok: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Trackcited by 31
  34. 2024BERGEN: A Benchmarking Library for Retrieval-Augmented Generationcited by 31
  35. 2024Few-shot Personalization of LLMs with Mis-aligned Responsescited by 30
  36. 2024Learning to Plan and Generate Text with Citationscited by 28
  37. 2025LongProc: Benchmarking Long-Context Language Models on Long Procedural Generationcited by 28
  38. 2024Chain-of-Action: Faithful and Multimodal Question Answering through Large Language Modelscited by 27

50 most cited of 231 All 231 (JSON)