Cost-Effective Agent Harnesses for Abstract Reasoning and Generalization on ARC-AGI-1
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Recent progress on ARC-AGI-1 from disclosed architectures has come broadly from two regimes: heavy test-time compute over frontier models (evolutionary search, exhaustive sampling, extended chain-of-thought), or benchmark-specific training in which small models are fine-tuned on ARC data, often with task-specialized architectures. We study a third regime: an open-weight model in non-thinking mode (DeepSeek V3.2) under a strict budget, with no ARC-specific fine-tuning. We study what is recoverable through architecture alone, building agentic harnesses that decompose pattern-discovery and program-synthesis stages explicitly. First, we introduce an Explorer-Definer Pipeline that separates pattern discovery from executable transformation synthesis, implemented as a two-stage agent pipeline. Next, we present the Reflective Orchestrator, which augments the pipeline with autonomous exploration of new transformations when previous hypotheses fail on training pairs. On the ARC-AGI-1 public 400-task evaluation set, the pipeline reaches 57.50% pass@2 at 0.25 per task, and the orchestrator reaches 67.25% pass@2 at 0.62 per task. Together these architectures lift a 15.50% one-shot baseline by ~52 points without benchmark-specific training or heavy test-time compute. Furthermore, the orchestrator-driven lift tests a falsifiable diagnostic the pipeline produces; unbiased pass@k analysis suggests the pipeline is generation-bound, not selection-bound (selection via training-pair accuracy captures ~95% of the candidate ceiling) and predicts that significant improvement requires broader generation, not better ranking. The orchestrator implements this prediction via adaptive re-exploration and confirms it (unbiased pass@1 lift +9.81 pp, matching selection-mediated pass@2 lift). An additional pipeline ablation identifies its think tool as a significant component, with removal reducing pass@2 by 5.75 pp.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 12 of its 23 references that are in the map. First posted on arXiv 7 Jul 2026.
Builds on
12 of 23 references.
Closest ancestors · 2
Same direction, newest first
- 2025Tiny Recursive Models on ARC-AGI-1: Inductive Biases, Identity Conditioning, and Test-Time Compute4 Dec
- 2025ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems17 May
Other references · 10
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Let's Verify Step by Stepcited by 3,789
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
- 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
- 2023Universal Self-Consistency for Large Language Model Generationcited by 203
- 2025Scalable Best-of-N Selection for Large Language Models via Self-Certaintycited by 187
- 2021Communicating Natural Programs to Humans and Machinescited by 35
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence
- 2025ARC-NCA: Towards Developmental Solutions to the Abstraction and Reasoning Corpus
- 2026A case study of evaluating AI agents on a neuroscience data-to-discovery pipeline
- 2026Auto: The AGI Compiler
- 2025Graph-Based Exploration for ARC-AGI-3 Interactive Reasoning Tasks
- 2026ANDES: Agent Native Data Evolving Synthesis Tool for Autonomous Instruction Alignment
- 2025Don't throw the baby out with the bathwater: How and why deep learning for ARC
- 2025Product of Experts with LLMs: Boosting Performance on ARC Is a Matter of Perspective