Modality-Driven Search with Holistic Trace Judging for ARC-AGI-2
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Large language models can produce fluent, internally coherent reasoning traces for abstract reasoning tasks while still being confidently wrong - making selection among candidates, not just generation, the central challenge. I present a solver for ARC-AGI-2, a few-shot visual reasoning benchmark, built around two principles: (i) treating reasoning modalities as search operators, generating diverse candidates independently across text, image, and code channels, and (ii) context-preserving holistic judging, in which a judge model jointly compares all candidate reasoning traces within a single long-context prompt. Unlike self-consistency or majority voting, this approach reliably recovers correct minority hypotheses on tasks where the modal answer is wrong. On the ARC Prize semi-private evaluation set, the solver achieves 72.9 percent at USD 38.99 per task - the highest score on the verified leaderboard at the time of writing, exceeding the best standalone frontier models, GPT-5.2 Pro at 54.2 percent and Gemini 3 Pro at 54.0 percent, by +18.7 percentage points. On the public evaluation set, it achieves 76.1 percent at USD 19.69 per task. I release the full source code and document extensive negative results, including the finding that prescriptive prompting templates and iterative refinement systematically reduce hypothesis diversity and degrade performance.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 16 of its 21 references that are in the map. First posted on arXiv 30 Jun 2026.
Builds on
16 of 21 references.
Closest ancestors · 2
Same direction, newest first
- 2024ARC Prize 2024: Technical Report5 Dec
- 2024Combining Induction and Transduction for Abstract Reasoning4 Nov
Other references · 14
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
- 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
- 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 886
- 2022PAL: Program-aided Language Modelscited by 652
- 2024The Surprising Effectiveness of Test-Time Training for Few-Shot Learningcited by 96
- 2024Addressing the Abstraction and Reasoning Corpus via Procedural Example Generationcited by 33
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026ARC-TGI: Human-Validated Task Generators with Reasoning Chain Templates for ARC-AGI
- 2025Tiny Recursive Models on ARC-AGI-1: Inductive Biases, Identity Conditioning, and Test-Time Compute
- 2025Your Reasoning Benchmark May Not Test Reasoning: Revealing Perception Bottleneck in Abstract Reasoning Benchmarks
- 2026DiARC: Distinguishing Positive and Negative Samples Helps Improving ARC-like Reasoning Ability of Large Language Models
- 2025Product of Experts with LLMs: Boosting Performance on ARC Is a Matter of Perspective
- 2026A Systematic Study of Small Language Models on Abstract Reasoning Tasks
- 2025Do AI Models Perform Human-like Abstract Reasoning Across Modalities?
- 2026StemBind: When MLLMs Get Lost Between Rules and Instances in Abstract Visual Reasoning