mlsanityData to 7 Oct 2026

Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge

arXiv 1803.05457 ↗First posted 14 Mar 2018cs.AI cs.CL cs.IRcited by 4,710

Not on the map. Why?

Cited by 615 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a new question set, text corpus, and baselines assembled to encourage AI research in advanced question answering. Together, these constitute the AI2 Reasoning Challenge (ARC), which requires far more powerful knowledge and reasoning than previous challenges such as SQuAD or SNLI. The ARC question set is partitioned into a Challenge Set and an Easy Set, where the Challenge Set contains only questions answered incorrectly by both a retrieval-based algorithm and a word co-occurence algorithm. The dataset contains only natural, grade-school science questions (authored for human tests), and is the largest public-domain set of this kind (7,787 questions). We test several baselines on the Challenge Set, including leading neural models from the SQuAD and SNLI tasks, and find that none are able to significantly outperform a random baseline, reflecting the difficult nature of this task. We are also releasing the ARC Corpus, a corpus of 14M science sentences relevant to the task, and implementations of the three neural baseline models tested. Can your model perform better? We pose ARC as a challenge to the community.

Key facts. Cited by 4,710 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 51; 2022: 71; 2023: 274; 2024: 1160; 2025: 1548; 2026: 1606.

4,710 papers.

Sort:

By area:

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2024Qwen2.5 Technical Reportcited by 4,713
  8. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  9. 2024GPT-4o System Cardcited by 4,467
  10. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  11. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  12. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
38 more
  1. 2023Qwen Technical Reportcited by 3,273
  2. 2023Mistral 7Bcited by 3,065
  3. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  4. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  5. 2023A Survey of Large Language Modelscited by 2,537
  6. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  7. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  8. 2024Qwen2 Technical Reportcited by 2,076
  9. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  10. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  11. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  12. 2024Mixtral of Expertscited by 1,785
  13. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  14. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  15. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  16. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,283
  17. 2023PaLM 2 Technical Reportcited by 1,154
  18. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  19. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  20. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  21. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  22. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  23. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  24. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  25. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  26. 2023BloombergGPT: A Large Language Model for Financecited by 923
  27. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  28. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  29. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
  30. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  31. 2025Large Language Diffusion Modelscited by 872
  32. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  33. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  34. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  35. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
  36. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  37. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  38. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755

50 most cited of 4,710 All 4,710 (JSON)