# Compositional diversity in visual concept learning

arXiv: https://arxiv.org/abs/2305.19374 · HTML: https://mlsanity.com/paper/2305.19374/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 55 references that are in the map.
- First posted on arXiv 30 May 2023.

## Builds on (most cited first, 15 of 15)

- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [When and why vision-language models behave like bags-of-words, and what to do about it?](https://mlsanity.com/paper/2210.01936/) (2022)
- [Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality](https://mlsanity.com/paper/2204.03162/) (2022)
- [CREPE: Can Vision-Language Foundation Models Reason Compositionally?](https://mlsanity.com/paper/2212.07796/) (2022)
- [Learning to Infer Graphics Programs from Hand-Drawn Images](https://mlsanity.com/paper/1707.09627/) (2017)
- [The Omniglot challenge: a 3-year progress report](https://mlsanity.com/paper/1902.03477/) (2019)
- [Human few-shot learning of compositional instructions](https://mlsanity.com/paper/1901.04587/) (2019)
- [Modeling rapid language learning by distilling Bayesian priors into artificial neural networks](https://mlsanity.com/paper/2305.14701/) (2023)
- [On the Quantitative Analysis of Decoder-Based Generative Models](https://mlsanity.com/paper/1611.04273/) (2016)
- [People infer recursive visual concepts from just a few examples](https://mlsanity.com/paper/1904.08034/) (2019)
- [Learning abstract structure for drawing by efficient motor program induction](https://mlsanity.com/paper/2008.03519/) (2020)
- [Learning to learn generative programs with Memoised Wake-Sleep](https://mlsanity.com/paper/2007.03132/) (2020)
- [Learning Task-General Representations with Generative Neuro-Symbolic Modeling](https://mlsanity.com/paper/2006.14448/) (2020)

## Built on by (most cited first, 8 of 8)

- [When does compositional structure yield compositional generalization? A kernel theory](https://mlsanity.com/paper/2405.16391/) (2024)
- [Skews in the Phenomenon Space Hinder Generalization in Text-to-Image Generation](https://mlsanity.com/paper/2403.16394/) (2024)
- [From Frege to chatGPT: Compositionality in language, cognition, and deep neural networks](https://mlsanity.com/paper/2405.15164/) (2024)
- [Whither symbols in the era of advanced neural networks?](https://mlsanity.com/paper/2508.05776/) (2025)
- [Compositional learning of functions in humans and machines](https://mlsanity.com/paper/2403.12201/) (2024)
- [Composing Linear Layers from Irreducibles](https://mlsanity.com/paper/2507.11688/) (2025)
- [Path-dependent program induction under resource constraints explains human sequence learning](https://mlsanity.com/paper/2606.20623/) (2026)
- [Gated Target Propagation for Compositional Generalization in Continual Learning](https://mlsanity.com/paper/2610.04649/) (2026)

## Abstract (arXiv)

Humans leverage compositionality to efficiently learn new concepts, understanding how familiar parts can combine together to form novel objects. In contrast, popular computer vision models struggle to make the same types of inferences, requiring more data and generalizing less flexibly than people do. Here, we study these distinctively human abilities across a range of different types of visual composition, examining how people classify and generate ``alien figures'' with rich relational structure. We also develop a Bayesian program induction model which searches for the best programs for generating the candidate visual figures, utilizing a large program space containing different compositional mechanisms and abstractions. In few shot classification tasks, we find that people and the program induction model can make a range of meaningful compositional generalizations, with the model providing a strong account of the experimental data as well as interpretable parameters that reveal human assumptions about the factors invariant to category membership (here, to rotation and changing part attachment). In few shot generation tasks, both people and the models are able to construct compelling novel examples, with people behaving in additional structured ways beyond the model capabilities, e.g. making choices that complete a set or reconfiguring existing parts in highly novel ways. To capture these additional behavioral patterns, we develop an alternative model based on neuro-symbolic program induction: this model also composes new concepts from existing parts yet, distinctively, it utilizes neural network modules to successfully capture residual statistical structure. Together, our behavioral and computational findings show how people and models can produce a rich variety of compositional behavior when classifying and generating visual objects.