mlsanityData to 14 Oct 2026

CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge

arXiv 1811.00937 ↗First posted 2 Nov 2018cs.CL cs.AI cs.LGcited by 1,822

Not on the map. Why?

Cited by 428 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

When answering a question, people often draw upon their rich world knowledge in addition to the particular context. Recent work has focused primarily on answering questions given some relevant document or context, and required very little general background. To investigate question answering with prior knowledge, we present CommonsenseQA: a challenging new dataset for commonsense question answering. To capture common sense beyond associations, we extract from ConceptNet (Speer et al., 2017) multiple target concepts that have the same semantic relation to a single source concept. Crowd-workers are asked to author multiple-choice questions that mention the source concept and discriminate in turn between each of the target concepts. This encourages workers to create questions with complex semantics that often require prior knowledge. We create 12,247 questions through this procedure and demonstrate the difficulty of our task with a large number of strong baselines. Our best baseline is based on BERT-large (Devlin et al., 2018) and obtains 56% accuracy, well below human performance, which is 89%.

Key facts. Cited by 1,822 papers in the map (data through 14 Oct 2026). First posted on arXiv 2 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 103; 2022: 134; 2023: 289; 2024: 401; 2025: 524; 2026: 371.

1,822 papers.

Sort:

By area:

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  7. 2023Qwen Technical Reportcited by 3,344
  8. 2023Mistral 7Bcited by 3,164
  9. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  10. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  11. 2023A Survey of Large Language Modelscited by 2,576
  12. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
38 more
  1. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  2. 2024Mixtral of Expertscited by 1,823
  3. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  4. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  5. 2023PaLM 2 Technical Reportcited by 1,162
  6. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
  7. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  8. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  9. 2023BloombergGPT: A Large Language Model for Financecited by 939
  10. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 916
  11. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  12. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
  13. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  14. 2022Galactica: A Large Language Model for Sciencecited by 744
  15. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
  16. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  17. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  18. 2022A Survey on In-context Learningcited by 721
  19. 2024Yi: Open Foundation Models by 01.AIcited by 719
  20. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
  21. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 688
  22. 2023A Comprehensive Overview of Large Language Modelscited by 648
  23. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 625
  24. 2022Towards Reasoning in Large Language Models: A Surveycited by 596
  25. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
  26. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 551
  27. 2023Active Retrieval Augmented Generationcited by 516
  28. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 488
  29. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 486
  30. 2021MetaICL: Learning to Learn In Contextcited by 454
  31. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 449
  32. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 406
  33. 2021QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answeringcited by 388
  34. 2023Detecting Pretraining Data from Large Language Modelscited by 387
  35. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 382
  36. 2023Language Models can Solve Computer Taskscited by 372
  37. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 368
  38. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 353

50 most cited of 1,822 All 1,822 (JSON)