mlsanityData to 7 Oct 2026

IIRC: A Dataset of Incomplete Information Reading Comprehension Questions

arXiv 2011.07127 ↗First posted 13 Nov 2020cs.CLcited by 52

Not on the map. Why?

Abstract (arXiv)

Humans often have to read multiple documents to address their information needs. However, most existing reading comprehension (RC) tasks only focus on questions for which the contexts provide all the information required to answer them, thus not evaluating a system's performance at identifying a potential lack of sufficient information and locating sources for that information. To fill this gap, we present a dataset, IIRC, with more than 13K questions over paragraphs from English Wikipedia that provide only partial information to answer them, with the missing information occurring in one or more linked documents. The questions were written by crowd workers who did not have access to any of the linked documents, leading to questions that have little lexical overlap with the contexts where the answers appear. This process also gave many questions without answers, and those that require discrete reasoning, increasing the difficulty of the task. We follow recent modeling work on various reading comprehension datasets to construct a baseline model for this dataset, finding that it achieves 31.1% F1 on this task, while estimated human performance is 88.4%. The dataset, code for the baseline system, and a leaderboard can be found at https://allennlp.org/iirc.

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 4; 2023: 14; 2024: 11; 2025: 8; 2026: 5.

52 papers.

Sort:

By area:

  1. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  2. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  3. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
  4. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  5. 2023Knowledge Graph Prompting for Multi-Document Question Answeringcited by 150
  6. 2024Benchmark Data Contamination of Large Language Models: A Surveycited by 138
  7. 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
  8. 2023Answering Questions by Meta-Reasoning over Multiple Chains of Thoughtcited by 87
  9. 2024DRAGIN: Dynamic Retrieval Augmented Generation based on the Information Needs of Large Language Modelscited by 63
  10. 2025A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challengescited by 58
  11. 2023The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Modelscited by 54
  12. 2024SeaKR: Self-aware Knowledge Retrieval for Adaptive Retrieval Augmented Generationcited by 40
38 more
  1. 2025Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMscited by 40
  2. 2021Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skillscited by 39
  3. 2023End-to-End Beam Retrieval for Multi-Hop Question Answeringcited by 35
  4. 2024Agent-Oriented Planning in Multi-Agent Systemscited by 34
  5. 2021ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answerscited by 29
  6. 2024TC-RAG:Turing-Complete RAG's Case study on Medical LLM Systemscited by 28
  7. 2021English Machine Reading Comprehension Datasets: A Surveycited by 26
  8. 2025SePer: Measure Retrieval Utility Through The Lens Of Semantic Perplexity Reductioncited by 24
  9. 2022Visconde: Multi-document QA with GPT-3 and Neural Rerankingcited by 23
  10. 2022Modeling Multi-hop Question Answering as Single Sequence Predictioncited by 21
  11. 2024Husky: A Unified, Open-Source Language Agent for Multi-Step Reasoningcited by 16
  12. 2021Exploiting Reasoning Chains for Multi-hop Science Question Answeringcited by 15
  13. 2021Break, Perturb, Build: Automatic Perturbation of Reasoning Paths Through Question Decompositioncited by 11
  14. 2024Empowering Language Models with Active Inquiry for Deeper Understandingcited by 11
  15. 2021Mitigating False-Negative Contexts in Multi-document Question Answering with Retrieval Marginalizationcited by 10
  16. 2024Graph Neural Network Enhanced Retrieval for Question Answering of LLMscited by 10
  17. 2023Drilling Down into the Discourse Structure with LLMs for Long Document Question Answeringcited by 8
  18. 2025Modeling Uncertainty Trends for Timely Retrieval in Dynamic RAGcited by 8
  19. 2022Teaching Broad Reasoning Skills for Multi-Step QA by Generating Hard Contextscited by 7
  20. 2024CuriousLLM: Elevating Multi-Document Question Answering with LLM-Enhanced Knowledge Graph Reasoningcited by 7
  21. 2025Accelerating Adaptive Retrieval Augmented Generation via Instruction-Driven Representation Reduction of Retrieval Overlapscited by 6
  22. 2025Optimizing Question Semantic Space for Dynamic Retrieval-Augmented Multi-hop Question Answeringcited by 6
  23. 2023Single Sequence Prediction over Reasoning Graphs for Multi-hop QAcited by 5
  24. 2023Mitigating the Impact of False Negatives in Dense Retrieval with Contrastive Confidence Regularizationcited by 4
  25. 2026KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasingcited by 4
  26. 2023KTRL+F: Knowledge-Augmented In-Document Searchcited by 3
  27. 2021Interactive Machine Comprehension with Dynamic Knowledge Graphscited by 2
  28. 2023Few-shot Unified Question Answering: Tuning Models or Prompts?cited by 2
  29. 2023Teaching Smaller Language Models To Generalise To Unseen Compositional Questionscited by 2
  30. 2023Towards leveraging LLMs for Conditional QAcited by 2
  31. 2024Leveraging Inter-Chunk Interactions for Enhanced Retrieval in Large Language Model-Based Question Answeringcited by 2
  32. 2023Answering Unseen Questions With Smaller Language Models Using Rationale Generation and Dense Retrievalcited by 1
  33. 2024QCG-Rerank: Chunks Graph Rerank with Query Expansion in Retrieval-Augmented LLMs for Tourism Domaincited by 1
  34. 2025Memory-Aware and Uncertainty-Guided Retrieval for Multi-Hop Question Answeringcited by 1
  35. 2025A Query-Aware Multi-Path Knowledge Graph Fusion Approach for Enhancing Retrieval-Augmented Generation in Large Language Modelscited by 1
  36. 2026Scaling Multi-Hop Training Data via Graph-Constrained Path Selectioncited by 1
  37. 2026ENTLORE: A Graph-Grounded Benchmark for Latent Organizational Reasoning in Enterprise Question Answeringcited by 1
  38. 2023SciMRC: Multi-perspective Scientific Machine Reading Comprehensioncited by 0

50 most cited of 52 All 52 (JSON)