mlsanityData to 14 Oct 2026

ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence

arXiv 2603.24621 ↗First posted 24 Mar 2026cs.AIcited by 37

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

We introduce ARC-AGI-3, an interactive benchmark for studying agentic intelligence through novel, abstract, turn-based environments in which agents must explore, infer goals, build internal models of environment dynamics, and plan effective action sequences without explicit instructions. Like its predecessors ARC-AGI-1 and 2, ARC-AGI-3 focuses entirely on evaluating fluid adaptive efficiency on novel tasks, while avoiding language and external knowledge. ARC-AGI-3 environments only leverage Core Knowledge priors and are difficulty-calibrated via extensive testing with human test-takers. Our testing shows humans can solve 100% of the environments, in contrast to frontier AI systems which, as of March 2026, score below 1%. In this paper, we present the benchmark design, its efficiency-based scoring framework grounded in human action baselines, and the methodology used to construct, validate, and calibrate the environments.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 3 of its 15 references that are in the map. First posted on arXiv 24 Mar 2026.

Builds on

3 of 15 references.

References · 3

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2018On Evaluation of Embodied Navigation Agentscited by 547

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 37.

37 papers.

Sort:

By area:

  1. 2026Rethinking the Evaluation of Harness Evolution for Agentscited by 48
  2. 2026PRO-LONG: Programmatic Memory Enables Long-Horizon Reasoningcited by 10
  3. 2026OPINE-World: Programmatic World Modeling with Ontology-error-Prioritized Interactive Exploration for ARC-AGI-3cited by 8
  4. 2026Workspace Optimization: How to Train Your Agentcited by 7
  5. 2026Tycho: Active Abstraction with Programmatic World Models for ARC-AGI-3cited by 7
  6. 2026Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world modelscited by 7
  7. 2026Twin: Playing an Unknown Game with a Test-Time Digital Twincited by 4
  8. 2026Odysseus: Scaling VLMs to 100+ Turn Decision-Making in Games via Reinforcement Learningcited by 3
  9. 2026Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiversecited by 3
  10. 2026Do Coding Agents Need Executable World Models, Simplification, and Verification to Solve ARC-AGI-3?cited by 3
  11. 2026Beyond Simply Environment Scaling: Designing Effective Environment Distributions for Multimodal Agent Learningcited by 3
  12. 2026AI for Games in the Foundation Model Eracited by 3
25 more
  1. 2026MAP: A Map-then-Act Paradigm for Long-Horizon Interactive Agent Reasoningcited by 2
  2. 2026Explore Before You Solve: The Speed--Depth Trade-off in Epistemic Agents for ARC-AGI-3cited by 2
  3. 2026Mind the Gap Between Spatial Reasoning and Acting! Step-by-Step Evaluation of Agents With Spatial-Gymcited by 1
  4. 2026Agentick: A Unified Benchmark for General Sequential Decision-Making Agentscited by 1
  5. 2026Reason to Play: Behavioral and Brain Alignment Between Frontier LRMs and Human Game Learnerscited by 1
  6. 2026GIM: Evaluating models via tasks that integrate multiple cognitive domainscited by 1
  7. 2026Playing ZendoWorld: Challenging AI Agents on Active Visual Concept Inductioncited by 1
  8. 2026EMPIRIC: Experiment-Driven Learning of Residual World Models for Robot Planningcited by 1
  9. 2026Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenescited by 1
  10. 2026VISTA: A Visual Harness for Reasoning in an Interactive Worldcited by 1
  11. 2026The Boundaries of Automation: A Theory of Persistent Human Participationcited by 0
  12. 2026Benchmark-Based Comparative Assessment of Publicly Benchmarked Indian Foundation Models: A Capability and Evaluation-Maturity Frameworkcited by 0
  13. 2026Framework and Benchmark for Code-Driven Agentic Testing in Web Developmentcited by 0
  14. 2026GameBoyWorlds: A Testbed for Self-Improvement in Embodied Video Gamescited by 0
  15. 2026Witness: Discovery, Deciphering, and Epiphany in Interactive Puzzle Environmentscited by 0
  16. 2026Schema: Discovering Unknown Environments via Agentic Program Inductioncited by 0
  17. 2026On-the-fly Weight Generation: A Hypernetwork Proof of Concept on ARC-1Dcited by 0
  18. 2026Kepler: Auditable World Models for ARC-AGI-3cited by 0
  19. 2026ULTRADISCOVERY: Abductive Exploration in an Interconnected, Epistemically Open Universecited by 0
  20. 2026PlaySuite: A Large-Scale Benchmark for Interactive Visual Intelligencecited by 0
  21. 2026SpeedrunBench: Challenging LLM Agents with Video Game Speedrunningcited by 0
  22. 2026Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments?cited by 0
  23. 2026A Systematic Study of Small Language Models on Abstract Reasoning Taskscited by 0
  24. 2026FreeEvolve: Learning to Evolve Beyond Fixed Loopscited by 0
  25. 2026Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebookscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems
  2. 2025Graph-Based Exploration for ARC-AGI-3 Interactive Reasoning Tasks
  3. 2026Cost-Effective Agent Harnesses for Abstract Reasoning and Generalization on ARC-AGI-1
  4. 2025Beyond Black-Box Benchmarking: Observability, Analytics, and Optimization of Agentic Systems
  5. 2026Running the Gauntlet: Challenging Agentic Tasks
  6. 2026Executable World Models for ARC-AGI-3 in the Era of Coding Agents
  7. 2026AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts
  8. 2026Atria Dawn: The Dawn of Agentic Superintelligence