# ARC-AGI Without Pretraining

arXiv: https://arxiv.org/abs/2512.06104 · HTML: https://mlsanity.com/paper/2512.06104/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 61 references that are in the map.
- First posted on arXiv 5 Dec 2025.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [Neural Turing Machines](https://mlsanity.com/paper/1410.5401/) (2014)
- [NVAE: A Deep Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2007.03898/) (2020)
- [Ladder Variational Autoencoders](https://mlsanity.com/paper/1602.02282/) (2016)
- [Hierarchical Reasoning Model](https://mlsanity.com/paper/2506.21734/) (2025)
- [A Brief Review of Hypernetworks in Deep Learning](https://mlsanity.com/paper/2306.06955/) (2023)
- [The Surprising Effectiveness of Test-Time Training for Few-Shot Learning](https://mlsanity.com/paper/2411.07279/) (2024)
- [Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameters](https://mlsanity.com/paper/1810.00440/) (2018)
- [Combining Induction and Transduction for Abstract Reasoning](https://mlsanity.com/paper/2411.02272/) (2024)
- [Compressing Images by Encoding Their Latent Representations with Relative Entropy Coding](https://mlsanity.com/paper/2010.01185/) (2020)
- [H-ARC: A Robust Estimate of Human Performance on the Abstraction and Reasoning Corpus Benchmark](https://mlsanity.com/paper/2409.01374/) (2024)
- [Searching Latent Program Spaces](https://mlsanity.com/paper/2411.08706/) (2024)
- [Don't throw the baby out with the bathwater: How and why deep learning for ARC](https://mlsanity.com/paper/2506.14276/) (2025)
- [An Alternative Practice of Tropical Convolution to Traditional Convolutional Neural Networks](https://mlsanity.com/paper/2103.02096/) (2021)
- [MADIL: An MDL-based Framework for Efficient Program Synthesis in the ARC Benchmark](https://mlsanity.com/paper/2505.01081/) (2025)

## Built on by (most cited first, 3 of 3)

- [PRO-LONG: Programmatic Memory Enables Long-Horizon Reasoning](https://mlsanity.com/paper/2607.20064/) (2026)
- [Agentic Frameworks for Reasoning Tasks: An Empirical Study](https://mlsanity.com/paper/2604.16646/) (2026)
- [From Global to Factor-Wise Expert Composition in Discrete Diffusion Models](https://mlsanity.com/paper/2607.11758/) (2026)

## Abstract (arXiv)

Conventional wisdom in the age of LLMs dictates that solving IQ-test-like visual puzzles from the ARC-AGI-1 benchmark requires capabilities derived from massive pretraining. To counter this, we introduce CompressARC, a 76K parameter model without any pretraining that solves 20% of evaluation puzzles by minimizing the description length (MDL) of the target puzzle purely during inference time. The MDL endows CompressARC with extreme generalization abilities typically unheard of in deep learning. To our knowledge, CompressARC is the only deep learning method for ARC-AGI where training happens only on a single sample: the target inference puzzle itself, with the final solution information removed. Moreover, CompressARC does not train on the pre-provided ARC-AGI "training set". Under these extremely data-limited conditions, we do not ordinarily expect any puzzles to be solvable at all. Yet CompressARC still solves a diverse distribution of creative ARC-AGI puzzles, suggesting MDL to be an alternative feasible way to produce intelligence, besides conventional pretraining.