QuAC : Question Answering in Context
Not on the map. Why?
Abstract (arXiv)
We present QuAC, a dataset for Question Answering in Context that contains 14K information-seeking QA dialogs (100K questions in total). The dialogs involve two crowd workers: (1) a student who poses a sequence of freeform questions to learn as much as possible about a hidden Wikipedia text, and (2) a teacher who answers the questions by providing short excerpts from the text. QuAC introduces challenges not found in existing machine comprehension datasets: its questions are often more open-ended, unanswerable, or only meaningful within the dialog context, as we show in a detailed qualitative evaluation. We also report results for a number of reference models, including a recently state-of-the-art reading comprehension architecture extended to model dialog context. Our best model underperforms humans by 20 F1, suggesting that there is significant room for future work on this data. Dataset, baseline, and leaderboard available at http://quac.ai.
Key facts. Cited by 380 papers in the map (data through 13 Oct 2026). First posted on arXiv 21 Aug 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 81; 2022: 70; 2023: 78; 2024: 87; 2025: 46; 2026: 18.
380 papers.
Sort:
By area:
- 2022Training language models to follow instructions with human feedbackcited by 15,856
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,979
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,765
- 2023Mistral 7Bcited by 3,123
- 2024Mixtral of Expertscited by 1,811
- 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 896
- 2024Yi: Open Foundation Models by 01.AIcited by 715
- 2023A Comprehensive Overview of Large Language Modelscited by 644
- 2024Large Language Models: A Surveycited by 520
- 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 478
- 2025LLMs Get Lost In Multi-Turn Conversationcited by 418
- 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 411
38 more
- 2024Jamba: A Hybrid Transformer-Mamba Language Modelcited by 394
- 2021Measuring Attribution in Natural Language Generation Modelscited by 253
- 2022Efficient Training of Language Models to Fill in the Middlecited by 251
- 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 231
- 2025Reasoning with Latent Thoughts: On the Power of Looped Transformerscited by 228
- 2022ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answeringcited by 196
- 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
- 2023Benchmarking Foundation Models with Language-Model-as-an-Examinercited by 172
- 2021ExT5: Towards Extreme Multi-Task Scaling for Transfer Learningcited by 161
- 2023FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learningcited by 149
- 2021SituatedQA: Incorporating Extra-Linguistic Contexts into QAcited by 145
- 2024Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluationscited by 140
- 2021Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformercited by 134
- 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
- 2023MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problemscited by 129
- 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 118
- 2022Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Modelscited by 109
- 2021WebSRC: A Dataset for Web-Based Structural Reading Comprehensioncited by 98
- 2021TopiOCQA: Open-domain Conversational Question Answering with Topic Switchingcited by 95
- 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
- 2024ChatQA: Surpassing GPT-4 on Conversational QA and RAGcited by 90
- 2023Context-faithful Prompting for Large Language Modelscited by 79
- 2024Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gapcited by 79
- 2021Evaluating Attribution in Dialogue Systems: The BEGIN Benchmarkcited by 76
- 2024SPIQA: A Dataset for Multimodal Question Answering on Scientific Paperscited by 75
- 2022CREPE: Open-Domain Question Answering with False Presuppositionscited by 71
- 2023Efficient Benchmarking of Language Modelscited by 71
- 2023Let the LLMs Talk: Simulating Human-to-Human Conversational QA via Zero-Shot LLM-to-LLM Interactionscited by 70
- 2023Enhancing Conversational Search: Large Language Model-Aided Informative Query Rewritingcited by 69
- 2021CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learningcited by 68
- 2024Instruction Pre-Training: Language Models are Supervised Multitask Learnerscited by 63
- 2023ConvGQR: Generative Query Reformulation for Conversational Searchcited by 59
- 2023LatestEval: Addressing Data Contamination in Language Model Evaluation through Dynamic and Time-Sensitive Test Constructioncited by 57
- 2022Dialog Inpainting: Turning Documents into Dialogscited by 56
- 2021MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documentscited by 53
- 2025MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systemscited by 50
- 2023An Open Source Data Contamination Report for Large Language Modelscited by 49
- 2023Estimating Contamination via Perplexity: Quantifying Memorisation in Language Model Evaluationcited by 47
50 most cited of 380 All 380 (JSON)
- 2026Evolution of Multimodal Question Answering: From Modality-Adaptive Extraction to Unified Language Representation8 Sep 2026
- 2026XYBench: Can LLMs Respond Pragmatically to Queries with Misconceptions?6 Sep 2026
- 2026Assessing Reliability of BERT-Based Models on Question Answering Tasks11 Aug 2026
- 2026CoinRAG: Contextualized Information Nugget KV Cache Reuse for Long-Context RAG7 Aug 2026
- 2026AILQA: Evaluating AI-Driven Legal Question Answering Systems for the Indian Legal System21 Jul 2026
- 2026Agentic Abstention: Do Agents Know When to Stop Instead of Act?27 Jun 2026
- 2026Every Eval Ever: A Unifying Schema and Community Repository for AI Evaluation Results12 Jun 2026
- 2026The Routing Plateau: Understanding the Accuracy Limits of LLM Routers27 May 2026
- 2026MTR-Suite: A Framework for Evaluating and Synthesizing Conversational Retrieval Benchmarks20 May 2026
- 2026LLM Benchmark Datasets Should Be Contamination-Resistant19 May 2026
- 2026Agentic Graph Retrieval-Augmented Generation for Auditable Commercial Registry Analysis15 Apr 2026
- 2026AgenticAI-DialogGen: Topic-Guided Conversation Generation for Fine-Tuning and Evaluating Short- and Long-Term Memories of LLMs14 Apr 2026
38 more
- 2026SinkTrack: Attention Sink based Context Anchoring for Large Language Models11 Apr 2026
- 2026DQA: Diagnostic Question Answering for IT Support7 Apr 2026
- 2026PassiveQA: A Three-Action Framework for Epistemically Calibrated Question Answering via Supervised Finetuning6 Apr 2026
- 2026When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturation18 Feb 2026
- 2026Comprehensive Comparison of RAG Methods Across Multi-Domain Conversational QA10 Feb 2026
- 2026SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass6 Feb 2026
- 2025Structured Definitions and Segmentations for Legal Reasoning in LLMs: A Study on Indian Legal Data14 Nov 2025
- 2025No Mean Feat: Simple, Strong Baselines for Context Compression23 Oct 2025
- 2025Multimedia-Aware Question Answering: A Review of Retrieval and Cross-Modal Reasoning Architectures23 Oct 2025
- 2025Mitigating Lost in Multi-turn Conversation via Curriculum RL with Verifiable Accuracy and Abstention Rewards21 Oct 2025
- 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications19 Oct 2025
- 2025A Longitudinal Study on Different Annotator Feedback Loops in Complex RAG Tasks13 Oct 2025
- 2025Prepared mind, fast response: A temporal decoupling framework for adaptive knowledge orchestration in open-domain dialogue9 Oct 2025
- 2025KnowMT-Bench: Benchmarking Knowledge-Intensive Long-Form Question Answering in Multi-Turn Dialogues26 Sep 2025
- 2025Explainable Semantic Text Relations: A Question-Answering Framework for Comparing Document Content10 Sep 2025
- 2025A Survey of the State-of-the-Art in Conversational Question Answering Systems6 Sep 2025
- 2025On Robustness and Reliability of Benchmark-Based Evaluation of LLMs4 Sep 2025
- 2025SelfAug: Mitigating Catastrophic Forgetting in Retrieval-Augmented Generation via Distribution Self-Alignment4 Sep 2025
- 2025Beyond Benchmarks: LLM Evaluation with an Anthropomorphic and Lifecycle-oriented Roadmap26 Aug 2025
- 2025Riemannian Optimization for LoRA on the Stiefel Manifold25 Aug 2025
- 2025An Entity Linking Agent for Question Answering5 Aug 2025
- 2025A Context-Aware Dual-Metric Framework for Confidence Estimation in Large Language Models1 Aug 2025
- 2025From Benchmarks to Skills: Low-Rank Factors for LLM Evaluation27 Jul 2025
- 2025Building Open-Retrieval Conversational Question Answering Systems by Generating Synthetic Data and Decontextualizing User Questions7 Jul 2025
- 2025Eka-Eval: An Evaluation Framework for Low-Resource Multilingual Large Language Models2 Jul 2025
- 2025TokenShapley: Token Level Context Attribution with Shapley Value18 Jun 2025
- 2025Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Quality17 Jun 2025
- 2025IP-Dialog: Evaluating Implicit Personalization in Dialogue Systems with Synthetic Data3 Jun 2025
- 2025textttDIAMONDs: A Dataset for mathbbDynamic mathbbInformation mathbbAnd mathbbMental modeling mathbbOf mathbbNumeric mathbbDiscussions19 May 2025
- 2025LLMs Get Lost In Multi-Turn Conversation9 May 2025
- 2025Assessing Robustness to Spurious Correlations in Post-Training Language Models9 May 2025
- 2025EnronQA: Towards Personalized RAG over Private Documents1 May 2025
- 2025Reassessing Extractive QA Datasets at Scale: LLM-as-a-Judge and In-Depth Analyses16 Apr 2025
- 2025Improving Instruct Models for Free: A Study on Partial Adaptation15 Apr 2025
- 2025Query Understanding in LLM-based Conversational Information Seeking8 Apr 2025
- 2025MLLM-Selector: Necessity and Diversity-driven High-Value Data Selection for Enhanced Visual Instruction Tuning26 Mar 2025
- 2025Understanding and Improving Information Preservation in Prompt Compression for LLMs24 Mar 2025
- 2025Efficient Intent-Based Filtering for Multi-Party Conversations Using Knowledge Distillation from LLMs21 Mar 2025
50 newest of 380 All 380 (JSON)
- 2024ChatQA: Surpassing GPT-4 on Conversational QA and RAGcited by 90
- 2025MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systemscited by 50
- 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applicationscited by 22
- 2024Entity Retrieval for Answering Entity-Centric Questionscited by 10
- 2025EnronQA: Towards Personalized RAG over Private Documentscited by 9
- 2025SelfAug: Mitigating Catastrophic Forgetting in Retrieval-Augmented Generation via Distribution Self-Alignmentcited by 9
- 2024Dynamic Contexts for Generating Suggestion Questions in RAG Based Conversational Systemscited by 7
- 2025Dialogue Benchmark Generation from Knowledge Graphs with Cost-Effective Retrieval-Augmented LLMscited by 7
- 2025Efficient OpAmp Adaptation for Zoom Attention to Golden Contextscited by 1
- 2026PassiveQA: A Three-Action Framework for Epistemically Calibrated Question Answering via Supervised Finetuningcited by 1
- 2026DQA: Diagnostic Question Answering for IT Supportcited by 1
- 2026Agentic Graph Retrieval-Augmented Generation for Auditable Commercial Registry Analysiscited by 1
12 most cited of 17 All 17 (JSON)
- 2026CoinRAG: Contextualized Information Nugget KV Cache Reuse for Long-Context RAG7 Aug 2026
- 2026MTR-Suite: A Framework for Evaluating and Synthesizing Conversational Retrieval Benchmarks20 May 2026
- 2026Agentic Graph Retrieval-Augmented Generation for Auditable Commercial Registry Analysis15 Apr 2026
- 2026DQA: Diagnostic Question Answering for IT Support7 Apr 2026
- 2026PassiveQA: A Three-Action Framework for Epistemically Calibrated Question Answering via Supervised Finetuning6 Apr 2026
- 2026Comprehensive Comparison of RAG Methods Across Multi-Domain Conversational QA10 Feb 2026
- 2025Multimedia-Aware Question Answering: A Review of Retrieval and Cross-Modal Reasoning Architectures23 Oct 2025
- 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications19 Oct 2025
- 2025A Longitudinal Study on Different Annotator Feedback Loops in Complex RAG Tasks13 Oct 2025
- 2025SelfAug: Mitigating Catastrophic Forgetting in Retrieval-Augmented Generation via Distribution Self-Alignment4 Sep 2025
- 2025EnronQA: Towards Personalized RAG over Private Documents1 May 2025
- 2025Efficient OpAmp Adaptation for Zoom Attention to Golden Contexts18 Feb 2025
12 newest of 17 All 17 (JSON)
- 2025Reasoning with Latent Thoughts: On the Power of Looped Transformerscited by 228
- 2022ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answeringcited by 196
- 2023MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problemscited by 129
- 2024Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gapcited by 79
- 2024Can Language Models Perform Robust Reasoning in Chain-of-thought Prompting with Noisy Rationales?cited by 41
- 2025LongFaith: Enhancing Long-Context Reasoning in LLMs with Faithful Synthetic Datacited by 16
- 2024Structured Chain-of-Thought Prompting for Few-Shot Generation of Content-Grounded QA Conversationscited by 9
- 2023Commonsense Reasoning for Conversational AI: A Survey of the State of the Artcited by 7
- 2023Few-shot Unified Question Answering: Tuning Models or Prompts?cited by 2
- 2024Ollabench: Evaluating LLMs' Reasoning for Human-centric Interdependent Cybersecuritycited by 2
- 2023Modeling Hierarchical Reasoning Chains by Linking Discourse Units and Key Phrases for Reading Comprehensioncited by 1
- 2024Large Language Models Need Consultants for Reasoning: Becoming an Expert in a Complex Human System Through Behavior Simulationcited by 1
12 most cited of 14 All 14 (JSON)
- 2025Structured Definitions and Segmentations for Legal Reasoning in LLMs: A Study on Indian Legal Data14 Nov 2025
- 2025Reasoning with Latent Thoughts: On the Power of Looped Transformers24 Feb 2025
- 2025LongFaith: Enhancing Long-Context Reasoning in LLMs with Faithful Synthetic Data18 Feb 2025
- 2024Can Language Models Perform Robust Reasoning in Chain-of-thought Prompting with Noisy Rationales?31 Oct 2024
- 2024LaMI: Augmenting Large Language Models via Late Multi-Image Fusion19 Jun 2024
- 2024Ollabench: Evaluating LLMs' Reasoning for Human-centric Interdependent Cybersecurity11 Jun 2024
- 2024Large Language Models Need Consultants for Reasoning: Becoming an Expert in a Complex Human System Through Behavior Simulation27 Mar 2024
- 2024Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gap29 Feb 2024
- 2024Structured Chain-of-Thought Prompting for Few-Shot Generation of Content-Grounded QA Conversations19 Feb 2024
- 2023Modeling Hierarchical Reasoning Chains by Linking Discourse Units and Key Phrases for Reading Comprehension21 Jun 2023
- 2023Few-shot Unified Question Answering: Tuning Models or Prompts?23 May 2023
- 2023MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems23 May 2023
12 newest of 14 All 14 (JSON)
- 2022Training language models to follow instructions with human feedbackcited by 15,856
- 2023Safer-Instruct: Aligning Language Models with Automated Preference Datacited by 27
- 2024Ask Optimal Questions: Aligning Large Language Models with Retriever's Preference in Conversationcited by 23
- 2025Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Qualitycited by 14
- 2025Assessing Robustness to Spurious Correlations in Post-Training Language Modelscited by 4
- 2025Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Quality17 Jun 2025
- 2025Assessing Robustness to Spurious Correlations in Post-Training Language Models9 May 2025
- 2024Ask Optimal Questions: Aligning Large Language Models with Retriever's Preference in Conversation19 Feb 2024
- 2023Safer-Instruct: Aligning Language Models with Automated Preference Data15 Nov 2023
- 2022Training language models to follow instructions with human feedback4 Mar 2022
- 2024Mixtral of Expertscited by 1,811
- 2024Jamba: A Hybrid Transformer-Mamba Language Modelcited by 394
- 2024BAM! Just Like That: Simple and Efficient Parameter Upcycling for Mixture of Expertscited by 16
- 2024Nexus: Specialization meets Adaptability for Efficiently Training Mixture of Expertscited by 7
- 2024Expert Routing with Synthetic Data for Continual Learningcited by 0
- 2024Expert Routing with Synthetic Data for Continual Learning22 Dec 2024
- 2024Nexus: Specialization meets Adaptability for Efficiently Training Mixture of Experts28 Aug 2024
- 2024BAM! Just Like That: Simple and Efficient Parameter Upcycling for Mixture of Experts15 Aug 2024
- 2024Jamba: A Hybrid Transformer-Mamba Language Model28 Mar 2024
- 2024Mixtral of Experts8 Jan 2024
- 2024Book2Dial: Generating Teacher-Student Interactions from Textbooks for Cost-Effective Development of Educational Chatbotscited by 25
- 2024DiSCo: LLM Knowledge Distillation for Efficient Sparse Retrieval in Conversational Searchcited by 8
- 2022A Semi-supervised Learning Approach with Two Teachers to Improve Breakdown Identification in Dialoguescited by 3
- 2025Efficient Intent-Based Filtering for Multi-Party Conversations Using Knowledge Distillation from LLMscited by 0
- 2025Efficient Intent-Based Filtering for Multi-Party Conversations Using Knowledge Distillation from LLMs21 Mar 2025
- 2024DiSCo: LLM Knowledge Distillation for Efficient Sparse Retrieval in Conversational Search18 Oct 2024
- 2024Book2Dial: Generating Teacher-Student Interactions from Textbooks for Cost-Effective Development of Educational Chatbots5 Mar 2024
- 2022A Semi-supervised Learning Approach with Two Teachers to Improve Breakdown Identification in Dialogues22 Feb 2022
- 2023CONVERSER: Few-Shot Conversational Dense Retrieval with Synthetic Data Generationcited by 6
- 2024Enhancing Table Representations with LLM-powered Synthetic Data Generationcited by 3
- 2025On Synthesizing Data for Context Attribution in Question Answeringcited by 2
- 2025Building Open-Retrieval Conversational Question Answering Systems by Generating Synthetic Data and Decontextualizing User Questionscited by 0
- 2025Building Open-Retrieval Conversational Question Answering Systems by Generating Synthetic Data and Decontextualizing User Questions7 Jul 2025
- 2025On Synthesizing Data for Context Attribution in Question Answering21 Feb 2025
- 2024Enhancing Table Representations with LLM-powered Synthetic Data Generation4 Nov 2024
- 2023CONVERSER: Few-Shot Conversational Dense Retrieval with Synthetic Data Generation13 Sep 2023
- 2026Agentic Abstention: Do Agents Know When to Stop Instead of Act?cited by 14
- 2025An Entity Linking Agent for Question Answeringcited by 2
- 2026Agentic Abstention: Do Agents Know When to Stop Instead of Act?27 Jun 2026
- 2025An Entity Linking Agent for Question Answering5 Aug 2025
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,979
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,765
- 2023Mistral 7Bcited by 3,123
- 2024Yi: Open Foundation Models by 01.AIcited by 715
- 2023A Comprehensive Overview of Large Language Modelscited by 644
- 2024Large Language Models: A Surveycited by 520
- 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 478
- 2025LLMs Get Lost In Multi-Turn Conversationcited by 418
- 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 411
- 2021Measuring Attribution in Natural Language Generation Modelscited by 253
- 2022Efficient Training of Language Models to Fill in the Middlecited by 251
- 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 231
12 most cited of 321 All 321 (JSON)
- 2026Evolution of Multimodal Question Answering: From Modality-Adaptive Extraction to Unified Language Representation8 Sep 2026
- 2026XYBench: Can LLMs Respond Pragmatically to Queries with Misconceptions?6 Sep 2026
- 2026Assessing Reliability of BERT-Based Models on Question Answering Tasks11 Aug 2026
- 2026AILQA: Evaluating AI-Driven Legal Question Answering Systems for the Indian Legal System21 Jul 2026
- 2026Every Eval Ever: A Unifying Schema and Community Repository for AI Evaluation Results12 Jun 2026
- 2026The Routing Plateau: Understanding the Accuracy Limits of LLM Routers27 May 2026
- 2026LLM Benchmark Datasets Should Be Contamination-Resistant19 May 2026
- 2026AgenticAI-DialogGen: Topic-Guided Conversation Generation for Fine-Tuning and Evaluating Short- and Long-Term Memories of LLMs14 Apr 2026
- 2026SinkTrack: Attention Sink based Context Anchoring for Large Language Models11 Apr 2026
- 2026When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturation18 Feb 2026
- 2026SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass6 Feb 2026
- 2025Mitigating Lost in Multi-turn Conversation via Curriculum RL with Verifiable Accuracy and Abstention Rewards21 Oct 2025
12 newest of 321 All 321 (JSON)