ARC-AGI Without Pretraining
Not on the map. Why?
Abstract (arXiv)
Conventional wisdom in the age of LLMs dictates that solving IQ-test-like visual puzzles from the ARC-AGI-1 benchmark requires capabilities derived from massive pretraining. To counter this, we introduce CompressARC, a 76K parameter model without any pretraining that solves 20% of evaluation puzzles by minimizing the description length (MDL) of the target puzzle purely during inference time. The MDL endows CompressARC with extreme generalization abilities typically unheard of in deep learning. To our knowledge, CompressARC is the only deep learning method for ARC-AGI where training happens only on a single sample: the target inference puzzle itself, with the final solution information removed. Moreover, CompressARC does not train on the pre-provided ARC-AGI "training set". Under these extremely data-limited conditions, we do not ordinarily expect any puzzles to be solvable at all. Yet CompressARC still solves a diverse distribution of creative ARC-AGI puzzles, suggesting MDL to be an alternative feasible way to produce intelligence, besides conventional pretraining.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 20 of its 61 references that are in the map. First posted on arXiv 5 Dec 2025.
Builds on
20 of 61 references.
References · 20
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2017Neural Discrete Representation Learningcited by 4,650
- 2016Group Equivariant Convolutional Networkscited by 1,154
- 2020On Layer Normalization in the Transformer Architecturecited by 906
- 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
- 2014Neural Turing Machinescited by 630
- 2016Ladder Variational Autoencoderscited by 315
- 2025Hierarchical Reasoning Modelcited by 167
- 2023A Brief Review of Hypernetworks in Deep Learningcited by 102
- 2024The Surprising Effectiveness of Test-Time Training for Few-Shot Learningcited by 88
8 more
- 2018Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameterscited by 56
- 2024Combining Induction and Transduction for Abstract Reasoningcited by 53
- 2020Compressing Images by Encoding Their Latent Representations with Relative Entropy Codingcited by 36
- 2024H-ARC: A Robust Estimate of Human Performance on the Abstraction and Reasoning Corpus Benchmarkcited by 32
- 2024Searching Latent Program Spacescited by 20
- 2025Don't throw the baby out with the bathwater: How and why deep learning for ARCcited by 7
- 2021An Alternative Practice of Tropical Convolution to Traditional Convolutional Neural Networkscited by 4
- 2025MADIL: An MDL-based Framework for Efficient Program Synthesis in the ARC Benchmarkcited by 3
Built on by
Citing papers per year: 2026: 3.
3 papers.