mlsanityData to 13 Oct 2026

Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning

arXiv 1908.05803 ↗First posted 16 Aug 2019cs.CLcited by 100

Not on the map. Why?

Abstract (arXiv)

Machine comprehension of texts longer than a single sentence often requires coreference resolution. However, most current reading comprehension benchmarks do not contain complex coreferential phenomena and hence fail to evaluate the ability of models to resolve coreference. We present a new crowdsourced dataset containing more than 24K span-selection questions that require resolving coreference among entities in over 4.7K English paragraphs from Wikipedia. Obtaining questions focused on such phenomena is challenging, because it is hard to avoid lexical cues that shortcut complex reasoning. We deal with this issue by using a strong baseline model as an adversary in the crowdsourcing loop, which helps crowdworkers avoid writing questions with exploitable surface cues. We show that state-of-the-art reading comprehension models perform significantly worse than humans on this benchmark---the best model performance is 70.5 F1, while the estimated human performance is 93.4 F1.

Key facts. Cited by 100 papers in the map (data through 13 Oct 2026). First posted on arXiv 16 Aug 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 24; 2022: 27; 2023: 25; 2024: 16; 2025: 6; 2026: 2.

100 papers.

Sort:

By area:

  1. 2022Galactica: A Large Language Model for Sciencecited by 741
  2. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 697
  3. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 478
  4. 2021MetaICL: Learning to Learn In Contextcited by 452
  5. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 230
  6. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 201
  7. 2021Reframing Instructional Prompts to GPTk's Languagecited by 176
  8. 2021CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLPcited by 142
  9. 2023Document Understanding Dataset and Evaluation (DUDE)cited by 133
  10. 2022CommonsenseQA 2.0: Exposing the Limits of AI through Gamificationcited by 109
  11. 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
  12. 2024ChatQA: Surpassing GPT-4 on Conversational QA and RAGcited by 90
38 more
  1. 2021SpartQA: : A Textual Question Answering Benchmark for Spatial Reasoningcited by 75
  2. 2021What to Pre-Train on? Efficient Intermediate Task Selectioncited by 63
  3. 2021Few-Shot Question Answering by Pretraining Span Selectioncited by 62
  4. 2022UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Trainingcited by 57
  5. 2022Improving In-Context Few-Shot Learning via Self-Supervised Trainingcited by 57
  6. 2021Comparing Test Sets with Item Response Theorycited by 56
  7. 2023InstructRetro: Instruction Tuning post Retrieval-Augmented Pretrainingcited by 53
  8. 2022Don't Blame the Annotator: Bias Already Starts in the Annotation Instructionscited by 47
  9. 2022Reasoning Like Program Executorscited by 45
  10. 2023Pre-Training to Learn in Contextcited by 42
  11. 2021Why Machine Reading Comprehension Models Learn Shortcuts?cited by 40
  12. 2022CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negationcited by 38
  13. 2024SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Modelscited by 37
  14. 2021Controllable Neural Dialogue Summarization with Personal Named Entity Planningcited by 36
  15. 2021Coreference-Aware Dialogue Summarizationcited by 34
  16. 2021On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Studycited by 31
  17. 2024Multi-Task Inference: Can Large Language Models Follow Multiple Instructions at Once?cited by 29
  18. 2021Retrieval-guided Counterfactual Generation for QAcited by 28
  19. 2021MixQG: Neural Question Generation with Mixed Answer Typescited by 28
  20. 2024SimRAG: Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domainscited by 28
  21. 2022ConTinTin: Continual Learning from Task Instructionscited by 26
  22. 2024RepLiQA: A Question-Answering Dataset for Benchmarking LLMs on Unseen Reference Contentcited by 26
  23. 2021Exploring Universal Intrinsic Task Subspace via Prompt Tuningcited by 25
  24. 2022F-coref: Fast, Accurate and Easy to Use Coreference Resolutioncited by 24
  25. 2022ProQA: Structural Prompt-based Pre-training for Unified Question Answeringcited by 21
  26. 2022Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Modelcited by 20
  27. 2023LIQUID: A Framework for List Question Answering Dataset Generationcited by 19
  28. 2024Maverick: Efficient and Accurate Coreference Resolution Defying Recent Trendscited by 19
  29. 2022Prompt Tuning for Discriminative Pre-trained Language Modelscited by 17
  30. 2022RQUGE: Reference-Free Metric for Evaluating Question Generation by Answering the Questioncited by 17
  31. 2022Learning Instructions with Unlabeled Data for Zero-Shot Cross-Task Generalizationcited by 15
  32. 2023ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up?cited by 15
  33. 2023Prompting with Pseudo-Code Instructionscited by 14
  34. 2025Reassessing Extractive QA Datasets at Scale: LLM-as-a-Judge and In-Depth Analysescited by 14
  35. 2022Machine Reading, Fast and Slow: When Do Models "Understand" Language?cited by 13
  36. 2022Eliciting and Understanding Cross-Task Skills with Task-Level Mixture-of-Expertscited by 12
  37. 2022LogiGAN: Learning Logical Reasoning via Adversarial Pre-trainingcited by 11
  38. 2023Learning to Initialize: Can Meta Learning Improve Cross-task Generalization in Prompt Tuning?cited by 11

50 most cited of 100 All 100 (JSON)