mlsanityData to 7 Oct 2026

Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning

arXiv 1909.00277 ↗First posted 31 Aug 2019cs.CL cs.AIcited by 281

Not on the map. Why?

Cited by 54 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Understanding narratives requires reading between the lines, which in turn, requires interpreting the likely causes and effects of events, even when they are not mentioned explicitly. In this paper, we introduce Cosmos QA, a large-scale dataset of 35,600 problems that require commonsense-based reading comprehension, formulated as multiple-choice questions. In stark contrast to most existing reading comprehension datasets where the questions focus on factual and literal understanding of the context paragraph, our dataset focuses on reading between the lines over a diverse collection of people's everyday narratives, asking such questions as "what might be the possible reason of ...?", or "what would have happened if ..." that require reasoning beyond the exact text spans in the context. To establish baseline performances on Cosmos QA, we experiment with several state-of-the-art neural architectures for reading comprehension, and also propose a new architecture that improves over the competitive baselines. Experimental results demonstrate a significant gap between machine (68.4%) and human performance (94%), pointing to avenues for future research on commonsense machine comprehension. Dataset, code and leaderboard is publicly available at https://wilburone.github.io/cosmos.

Key facts. Cited by 281 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Aug 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 59; 2022: 57; 2023: 62; 2024: 56; 2025: 33; 2026: 14.

281 papers.

Sort:

By area:

  1. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  2. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  3. 2021Measuring Coding Challenge Competence With APPScited by 879
  4. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
  5. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  6. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  7. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  8. 2021MetaICL: Learning to Learn In Contextcited by 450
  9. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  10. 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 420
  11. 2022UL2: Unifying Language Learning Paradigmscited by 292
  12. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 246
38 more
  1. 2024LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Taskscited by 236
  2. 2021SPoT: Better Frozen Model Adaptation through Soft Prompt Transfercited by 235
  3. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
  4. 2021QuALITY: Question Answering with Long Input Texts, Yes!cited by 204
  5. 2021What Will it Take to Fix Benchmarking in Natural Language Understanding?cited by 200
  6. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 200
  7. 2022Leveraging Large Language Models for Multiple Choice Question Answeringcited by 199
  8. 2021Muppet: Massive Multi-task Representations with Pre-Finetuningcited by 192
  9. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  10. 2022Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Modelscited by 179
  11. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  12. 2021Reframing Instructional Prompts to GPTk's Languagecited by 176
  13. 2022ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoningcited by 175
  14. 2021ExT5: Towards Extreme Multi-Task Scaling for Transfer Learningcited by 161
  15. 2022A Review on Language Models as Knowledge Basescited by 156
  16. 2022Deep Bidirectional Language-Knowledge Graph Pretrainingcited by 156
  17. 2021CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLPcited by 142
  18. 2023Unified Demonstration Retriever for In-Context Learningcited by 141
  19. 2021Can Machines Learn Morality? The Delphi Experimentcited by 140
  20. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoningcited by 136
  21. 2024Localizing Task Information for Improved Model Merging and Compressioncited by 130
  22. 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
  23. 2024Benchmarking LLMs via Uncertainty Quantificationcited by 124
  24. 2023Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMscited by 103
  25. 2024Large Language Models Must Be Taught to Know What They Don't Knowcited by 99
  26. 2021UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmarkcited by 98
  27. 2022What Artificial Neural Networks Can Tell Us About Human Language Acquisitioncited by 97
  28. 2023Shepherd: A Critic for Language Model Generationcited by 96
  29. 2023SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoningcited by 94
  30. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 93
  31. 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
  32. 2024ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabiccited by 87
  33. 2023Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind Capabilitiescited by 86
  34. 2023Exploring the Benefits of Training Expert Language Models over Instruction Tuningcited by 80
  35. 2023ReCEval: Evaluating Reasoning Chains via Correctness and Informativenesscited by 76
  36. 2024Parameter Competition Balancing for Model Mergingcited by 71
  37. 2024LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Modelscited by 70
  38. 2023TIGERScore: Towards Building Explainable Metric for All Text Generation Taskscited by 69

50 most cited of 281 All 281 (JSON)