mlsanityData to 7 Oct 2026

CLAPNQ: Cohesive Long-form Answers from Passages in Natural Questions for RAG systems

arXiv 2404.02103 ↗First posted 2 Apr 2024cs.CLcited by 23

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Retrieval Augmented Generation (RAG) has become a popular application for large language models. It is preferable that successful RAG systems provide accurate answers that are supported by being grounded in a passage without any hallucinations. While considerable work is required for building a full RAG pipeline, being able to benchmark performance is also necessary. We present ClapNQ, a benchmark Long-form Question Answering dataset for the full RAG pipeline. ClapNQ includes long answers with grounded gold passages from Natural Questions (NQ) and a corpus to perform either retrieval, generation, or the full RAG pipeline. The ClapNQ answers are concise, 3x smaller than the full passage, and cohesive, meaning that the answer is composed fluently, often by integrating multiple pieces of the passage that are not contiguous. RAG models must adapt to these properties to be successful at ClapNQ. We present baseline experiments and analysis for ClapNQ that highlight areas where there is still significant room for improvement in grounded RAG. CLAPNQ is publicly available at https://github.com/primeqa/clapnq

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 30 of its 47 references that are in the map. First posted on arXiv 2 Apr 2024.

Builds on

30 of 47 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Benchmarking Large Language Models in Retrieval-Augmented Generation4 Sep

Other references · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2023Mistral 7Bcited by 3,065
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  8. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  9. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  10. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  11. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  12. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
17 more
  1. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  2. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  3. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  4. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  5. 2019ELI5: Long Form Question Answeringcited by 567
  6. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 408
  7. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 385
  8. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 231
  9. 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 230
  10. 2022Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Modelscited by 108
  11. 2023ExpertQA: Expert-Curated Questions and Attributed Answerscited by 96
  12. 2021TopiOCQA: Open-domain Conversational Question Answering with Topic Switchingcited by 95
  13. 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
  14. 2023RECALL: A Benchmark for LLMs Robustness against External Counterfactual Knowledgecited by 51
  15. 2020AQuaMuSe: Automatically Generating Datasets for Query-Based Multi-Document Summarizationcited by 42
  16. 2022QAMPARI: An Open-domain Question Answering Benchmark for Questions with Many Answers from Multiple Paragraphscited by 42
  17. 2019Joint Learning of Answer Selection and Answer Summary Generation in Community Question Answeringcited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 10; 2026: 5.

23 papers.

Sort:

By area:

  1. 2025MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systemscited by 50
  2. 2024Ragnar"ok: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Trackcited by 31
  3. 2024Question-Based Retrieval using Atomic Units for Enterprise RAGcited by 16
  4. 2025FreshStack: Building Realistic Benchmarks for Evaluating Retrieval on Technical Documentscited by 15
  5. 2024MIRAGE-Bench: Automatic Multilingual Benchmark Arena for Retrieval-Augmented Generation Systemscited by 14
  6. 2025CoCoA: Confidence and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in Large Language Modelscited by 9
  7. 2025Classifying and Addressing the Diversity of Errors in Retrieval-Augmented Generation Systemscited by 9
  8. 2024Do RAG Systems Cover What Matters? Evaluating and Optimizing Responses with Sub-Question Coveragecited by 8
  9. 2026MTRAG-UN: A Benchmark for Open Challenges in Multi-Turn RAG Conversationscited by 8
  10. 2024RAGProbe: An Automated Approach for Evaluating RAG Applicationscited by 7
  11. 2025Teaching Large Language Models to Maintain Contextual Faithfulness via Synthetic Tasks and Reinforcement Learningcited by 7
  12. 2024Stay Tuned: An Empirical Study of the Impact of Hyperparameters on LLM Tuning in Real-World Applicationscited by 6
11 more
  1. 2026Corpus2Skill: Distilling Enterprise Knowledge into Navigable Agent Skills for QA and RAGcited by 6
  2. 2024InspectorRAGet: An Introspection Platform for RAG Evaluationcited by 3
  3. 2025Reinforced Informativeness Optimization for Long-Form Retrieval-Augmented Generationcited by 2
  4. 2025Investigating Retrieval-Augmented Generation Systems on Unanswerable, Uncheatable, Realistic, Multi-hop Queriescited by 2
  5. 2024Measuring the Groundedness of Legal Question-Answering Systemscited by 1
  6. 2025An Analysis of Hyper-Parameter Optimization Methods for Retrieval Augmented Generationcited by 1
  7. 2026ACL-Verbatim: hallucination-free question answering for researchcited by 1
  8. 2025A Library of LLM Intrinsics for Retrieval-Augmented Generationcited by 0
  9. 2025RAGVUE: A Diagnostic View for Explainable and Automated Evaluation of Retrieval-Augmented Generationcited by 0
  10. 2026RAG-DIVE: A Dynamic Approach for Multi-Turn Dialogue Evaluation in Retrieval-Augmented Generationcited by 0
  11. 2026Can Virtual Agents Care? Designing an Empathetic and Personalized LLM-Driven Conversational Agentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024W-RAG: Weakly Supervised Dense Retrieval in RAG for Open-domain Question Answering
  2. 2024RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answering
  3. 2024In Defense of RAG in the Era of Long-Context Language Models
  4. 2024LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering
  5. 2024Long2RAG: Evaluating Long-Context & Long-Form Retrieval-Augmented Generation with Key Point Recall
  6. 2025RAGtifier: Evaluating RAG Generation Approaches of State-of-the-Art RAG Systems for the SIGIR LiveRAG Competition
  7. 2025SAGE: A Framework of Precise Retrieval for RAG
  8. 2025Structured RAG for Answering Aggregative Questions