# Compositional-ARC: Assessing Systematic Generalization in Abstract Spatial Reasoning

arXiv: https://arxiv.org/abs/2504.01445 · HTML: https://mlsanity.com/paper/2504.01445/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 12 of its 73 references that are in the map.
- First posted on arXiv 2 Apr 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 12 of 12)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [RL2: Fast Reinforcement Learning via Slow Reinforcement Learning](https://mlsanity.com/paper/1611.02779/) (2016)
- [On the Measure of Intelligence](https://mlsanity.com/paper/1911.01547/) (2019)
- [Addressing the Abstraction and Reasoning Corpus via Procedural Example Generation](https://mlsanity.com/paper/2404.07353/) (2024)
- [AlignVLM: Bridging Vision and Language Latent Spaces for Multimodal Document Understanding](https://mlsanity.com/paper/2502.01341/) (2025)
- [Towards Compositionally Generalizable Semantic Parsing in Large Language Models: A Survey](https://mlsanity.com/paper/2404.13074/) (2024)

## Built on by (most cited first, 4 of 4)

- [COGITAO: A Visual Reasoning Framework To Study Compositionality & Generalization](https://mlsanity.com/paper/2509.05249/) (2025)
- [From Global to Factor-Wise Expert Composition in Discrete Diffusion Models](https://mlsanity.com/paper/2607.11758/) (2026)
- [What Do Current Systematic Generalization Tasks Miss? A Reasoning-Centered Analysis](https://mlsanity.com/paper/2609.19212/) (2026)
- [Pretraining and Test-Time Training Budgets Matter for ARC: A Controlled Study of VARC and TRM](https://mlsanity.com/paper/2610.13660/) (2026)

## Abstract (arXiv)

Systematic generalization refers to the capacity to understand and generate novel combinations from known components. Despite recent progress by large language models (LLMs) across various domains, these models often fail to extend their knowledge to novel compositional scenarios, revealing notable limitations in systematic generalization. There has been an ongoing debate about whether neural networks possess the capacity for systematic generalization, with recent studies suggesting that meta-learning approaches designed for compositionality can significantly enhance this ability. However, these insights have largely been confined to linguistic problems, leaving their applicability to other tasks an open question. In this study, we extend meta-learning for compositionality to the domain of abstract spatial reasoning. To this end, we introduce textitCompositional-ARCunicodex2014a dataset designed to evaluate the capacity of models to systematically generalize from known geometric transformations (e.g., translation, rotation) of abstract two-dimensional objects to novel combinations of these transformations (e.g., translation+rotation). Our results show that a small transformer-based encoder-decoder model, trained via meta-learning for compositionality, can systematically generalize to previously unseen transformation compositions. Notably, despite having only 5.7M parameters, this model significantly outperforms state-of-the-art LLMsunicodex2014including o3-mini, GPT-4o, and Gemini 2.0 Flash, which fail to exhibit similar systematic behaviorunicodex2014and performs on par with the winning model of the ARC prize 2024, an 8B-parameter LLM trained via test-time training. Our findings highlight the effectiveness of meta-learning in promoting systematicity beyond linguistic tasks, suggesting a promising direction toward more robust and generalizable models.