mlsanityData to 7 Oct 2026

ARC Prize 2024: Technical Report

arXiv 2412.04604 ↗First posted 5 Dec 2024cs.AIcited by 103

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

As of December 2024, the ARC-AGI benchmark is five years old and remains unbeaten. We believe it is currently the most important unsolved AI benchmark in the world because it seeks to measure generalization on novel tasks -- the essence of intelligence -- as opposed to skill at tasks that can be prepared for in advance. This year, we launched ARC Prize, a global competition to inspire new ideas and drive open progress towards AGI by reaching a target benchmark score of 85%. As a result, the state-of-the-art score on the ARC-AGI private evaluation set increased from 33% to 55.5%, propelled by several frontier AGI reasoning techniques including deep learning-guided program synthesis and test-time training. In this paper, we survey top approaches, review new open-source implementations, discuss the limitations of the ARC-AGI-1 dataset, and share key insights gained from the competition.

Key facts. Cited by 103 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 5 of its 18 references that are in the map. First posted on arXiv 5 Dec 2024.

Builds on

5 of 18 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Combining Induction and Transduction for Abstract Reasoning4 Nov

Other references · 4

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2024H-ARC: A Robust Estimate of Human Performance on the Abstraction and Reasoning Corpus Benchmarkcited by 32
  3. 2024Searching Latent Program Spacescited by 20
  4. 2024Towards Efficient Neurally-Guided Program Induction for ARC-AGIcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 71; 2026: 32.

103 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systemscited by 171
  3. 2025Hierarchical Reasoning Modelcited by 167
  4. 2025xbench: Tracking Agents Productivity Scaling with Profession-Aligned Real-World Evaluationscited by 118
  5. 2025Is Best-of-N the Best of Them? Coverage, Scaling, and Optimality in Inference-Time Alignmentcited by 91
  6. 2025Zebra-CoT: A Dataset for Interleaved Vision Language Reasoningcited by 78
  7. 2025Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspectivecited by 75
  8. 2025The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMscited by 62
  9. 2025ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Modelscited by 56
  10. 2025Every Step Evolves: Scaling Reinforcement Learning for Trillion-Scale Thinking Modelcited by 50
  11. 2025CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Rewardcited by 39
  12. 2025Beyond GPT-5: Making LLMs Cheaper and Better via Performance-Efficiency Optimized Routingcited by 32
38 more
  1. 2025Limitations of Large Language Models in Clinical Problem-Solving Arising from Inflexible Reasoningcited by 29
  2. 2025Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learningcited by 28
  3. 2025We Can't Understand AI Using our Existing Vocabularycited by 25
  4. 2025MultiNRC: A Challenging and Native Multilingual Reasoning Evaluation Benchmark for LLMscited by 16
  5. 2025AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defensescited by 14
  6. 2026LoopViT: Scaling Visual ARC with Looped Transformerscited by 14
  7. 2025WebGames: Challenging General-Purpose Web-Browsing AI Agentscited by 12
  8. 2025BOTS: A Unified Framework for Bayesian Online Task Selection in LLM Reinforcement Finetuningcited by 12
  9. 2025RE-IMAGINE: Symbolic Benchmark Synthesis for Reasoning Evaluationcited by 11
  10. 2025Universal Reasoning Modelcited by 11
  11. 2026Jet-RL: Enabling On-Policy FP8 Reinforcement Learning with Unified Training and Rollout Precision Flowcited by 11
  12. 2025ARC-NCA: Towards Developmental Solutions to the Abstraction and Reasoning Corpuscited by 10
  13. 2025Do AI Models Perform Human-like Abstract Reasoning Across Modalities?cited by 10
  14. 2025Levels of Analysis for Large Language Modelscited by 9
  15. 2025Neural cellular automata: applications to biology and beyond classical AIcited by 9
  16. 2025Semantic Soft Bootstrapping: Long Context Reasoning in LLMs without Reinforcement Learningcited by 9
  17. 2025EconEvals: Benchmarks and Litmus Tests for Economic Decision-Making by LLM Agentscited by 8
  18. 2025Benchmarking LLMs' Swarm intelligencecited by 8
  19. 2025GVGAI-LLM: Evaluating Large Language Model Agents with Infinite Gamescited by 8
  20. 202511Plus-Bench: Demystifying Multimodal LLM Spatial Reasoning with Cognitive-Inspired Analysiscited by 8
  21. 2025From Generation to Generalization: Emergent Few-Shot Learning in Video Diffusion Modelscited by 7
  22. 2025Truly Assessing Fluid Intelligence of Large Language Models through Dynamic Reasoning Evaluationcited by 6
  23. 2025Which LLMs Get the Joke? Probing Non-STEM Reasoning Abilities with HumorBenchcited by 6
  24. 2025Test-Time Matching: Unlocking Compositional Reasoning in Multimodal Modelscited by 6
  25. 2026General365: Benchmarking General Reasoning in Large Language Models Across Diverse and Challenging Taskscited by 6
  26. 2026Ling and Ring 2.6 Technical Report: Efficient and Instant Agentic Intelligence at Trillion-Parameter Scalecited by 6
  27. 2025L0-Reasoning Bench: Evaluating Procedural Correctness in Language Models via Simple Program Executioncited by 5
  28. 2026OpenCompass: A Universal Evaluation Platform for Large Language Modelscited by 5
  29. 2025Offline Learning and Forgetting for Reasoning with Large Language Modelscited by 4
  30. 2025CausalARC: Abstract Reasoning with Causal World Modelscited by 4
  31. 2025A Fragile Number Sense: Probing the Elemental Limits of Numerical Reasoning in LLMscited by 4
  32. 2025PRISM-Bench: A Benchmark of Puzzle-Based Visual Tasks with CoT Error Detectioncited by 4
  33. 2026Transferability for General Reasoning: An Automated Curriculum for Multi-Domain RLVRcited by 4
  34. 2026Twin: Playing an Unknown Game with a Test-Time Digital Twincited by 4
  35. 2025Improving Rule-based Reasoning in LLMs using Neurosymbolic Representationscited by 3
  36. 2025Shape and Texture Recognition in Large Vision-Language Modelscited by 3
  37. 2025Lightweight Latent Verifiers for Efficient Meta-Generation Strategiescited by 3
  38. 2025On the Evaluation of Engineering Artificial General Intelligencecited by 3

50 most cited of 103 All 103 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence
  2. 2026Cost-Effective Agent Harnesses for Abstract Reasoning and Generalization on ARC-AGI-1
  3. 2025Product of Experts with LLMs: Boosting Performance on ARC Is a Matter of Perspective
  4. 2023Unraveling the ARC Puzzle: Mimicking Human Solutions with Object-Centric Decision Transformer
  5. 2025Graph-Based Exploration for ARC-AGI-3 Interactive Reasoning Tasks
  6. 2025Don't throw the baby out with the bathwater: How and why deep learning for ARC
  7. 2024AI Predicts AGI: Leveraging AGI Forecasting and Peer Review to Explore LLMs' Complex Reasoning Capabilities
  8. 2025Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models