mlsanityData to 14 Oct 2026

Modality-Driven Search with Holistic Trace Judging for ARC-AGI-2

arXiv 2606.31543 ↗First posted 30 Jun 2026cs.AI cs.CL cs.LGcited by 0

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Large language models can produce fluent, internally coherent reasoning traces for abstract reasoning tasks while still being confidently wrong - making selection among candidates, not just generation, the central challenge. I present a solver for ARC-AGI-2, a few-shot visual reasoning benchmark, built around two principles: (i) treating reasoning modalities as search operators, generating diverse candidates independently across text, image, and code channels, and (ii) context-preserving holistic judging, in which a judge model jointly compares all candidate reasoning traces within a single long-context prompt. Unlike self-consistency or majority voting, this approach reliably recovers correct minority hypotheses on tasks where the modal answer is wrong. On the ARC Prize semi-private evaluation set, the solver achieves 72.9 percent at USD 38.99 per task - the highest score on the verified leaderboard at the time of writing, exceeding the best standalone frontier models, GPT-5.2 Pro at 54.2 percent and Gemini 3 Pro at 54.0 percent, by +18.7 percentage points. On the public evaluation set, it achieves 76.1 percent at USD 19.69 per task. I release the full source code and document extensive negative results, including the finding that prescriptive prompting templates and iterative refinement systematically reduce hypothesis diversity and degrade performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 16 of its 21 references that are in the map. First posted on arXiv 30 Jun 2026.

Builds on

16 of 21 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024ARC Prize 2024: Technical Report5 Dec
  2. 2024Combining Induction and Transduction for Abstract Reasoning4 Nov

Other references · 14

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  6. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  8. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  9. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 886
  10. 2022PAL: Program-aided Language Modelscited by 652
  11. 2024The Surprising Effectiveness of Test-Time Training for Few-Shot Learningcited by 96
  12. 2024Addressing the Abstraction and Reasoning Corpus via Procedural Example Generationcited by 33
2 more
  1. 2021First Steps of an Approach to the ARC Challenge based on Descriptive Grid Models and the Minimum Description Length Principlecited by 13
  2. 2025ARC-GEN: A Mimetic Procedural Benchmark Generator for the Abstraction and Reasoning Corpuscited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ARC-TGI: Human-Validated Task Generators with Reasoning Chain Templates for ARC-AGI
  2. 2025Tiny Recursive Models on ARC-AGI-1: Inductive Biases, Identity Conditioning, and Test-Time Compute
  3. 2025Your Reasoning Benchmark May Not Test Reasoning: Revealing Perception Bottleneck in Abstract Reasoning Benchmarks
  4. 2026DiARC: Distinguishing Positive and Negative Samples Helps Improving ARC-like Reasoning Ability of Large Language Models
  5. 2025Product of Experts with LLMs: Boosting Performance on ARC Is a Matter of Perspective
  6. 2026A Systematic Study of Small Language Models on Abstract Reasoning Tasks
  7. 2025Do AI Models Perform Human-like Abstract Reasoning Across Modalities?
  8. 2026StemBind: When MLLMs Get Lost Between Rules and Instances in Abstract Visual Reasoning