mlsanityData to 7 Oct 2026

Compositional Generalization from First Principles

arXiv 2307.05596 ↗First posted 10 Jul 2023cs.LG stat.MLcited by 62

Not on the map. Why?

Abstract (arXiv)

Leveraging the compositional nature of our world to expedite learning and facilitate generalization is a hallmark of human perception. In machine learning, on the other hand, achieving compositional generalization has proven to be an elusive goal, even for models with explicit compositional priors. To get a better handle on compositional generalization, we here approach it from the bottom up: Inspired by identifiable representation learning, we investigate compositionality as a property of the data-generating process rather than the data itself. This reformulation enables us to derive mild conditions on only the support of the training distribution and the model architecture, which are sufficient for compositional generalization. We further demonstrate how our theoretical framework applies to real-world scenarios and validate our findings empirically. Our results set the stage for a principled theoretical study of compositional generalization.

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). Builds on 23 of its 37 references that are in the map. First posted on arXiv 10 Jul 2023.

Builds on

23 of 37 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  3. 2020Object-Centric Learning with Slot Attentioncited by 865
  4. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  5. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
  6. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  7. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 308
  8. 2019Measuring Compositional Generalization: A Comprehensive Method on Realistic Datacited by 280
  9. 2021Combinatorial optimization and reasoning with graph neural networkscited by 267
  10. 2021Conditional Object-Centric Learning from Videocited by 247
  11. 2021Contrastive Learning Inverts the Data Generating Processcited by 220
  12. 2019On the Fairness of Disentangled Representationscited by 144
11 more
  1. 2019Are Disentangled Representations Helpful for Abstract Visual Reasoning?cited by 100
  2. 2020Learning to Recombine and Resample Data for Compositional Generalizationcited by 65
  3. 2021Visual Representation Learning Does Not Generalize Strongly Within the Same Domaincited by 60
  4. 2019Compositional generalization in a deep seq2seq model by separating syntax and semanticscited by 56
  5. 2018Rearranging the Familiar: Testing Compositional Generalization in Recurrent Networkscited by 51
  6. 2022Complex-Valued Autoencoders for Object Discoverycited by 40
  7. 2018Exploiting Semantics in Neural Machine Translation with Graph Convolutional Networkscited by 30
  8. 2022Embrace the Gap: VAEs Perform Independent Mechanism Analysiscited by 28
  9. 2022First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domainscited by 20
  10. 2023Learning to Extrapolate: A Transductive Approachcited by 19
  11. 2023Learning and generalization of compositional representations of visual scenescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 19; 2025: 26; 2026: 12.

62 papers.

Sort:

By area:

  1. 2025WebSailor: Navigating Super-human Reasoning for Web Agentcited by 218
  2. 2023Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Taskcited by 91
  3. 2023Provable Compositional Generalization for Object-Centric Learningcited by 41
  4. 2024Compositional Generative Modeling: A Single Model is Not All You Needcited by 41
  5. 2023Does CLIP's Generalization Performance Mainly Stem from High Train-Test Similarity?cited by 35
  6. 2024Closing the Gap between TD Learning and Supervised Learning -- A Generalisation Point of Viewcited by 26
  7. 2024On Provable Length and Compositional Generalizationcited by 22
  8. 2024When does compositional structure yield compositional generalization? A kernel theorycited by 19
  9. 2024Interaction Asymmetry: A General Principle for Learning Composable Abstractionscited by 19
  10. 2024Deciphering the Role of Representation Disentanglement: Investigating Compositional Generalization in CLIP Modelscited by 13
  11. 2024How Diffusion Models Learn to Factorize and Composecited by 13
  12. 2024Compositional Risk Minimizationcited by 13
38 more
  1. 2024A Complexity-Based Theory of Compositionalitycited by 13
  2. 2025Mechanisms of Projective Composition of Diffusion Modelscited by 13
  3. 2025Task Generalization With AutoRegressive Compositional Structure: Can Learning From D Tasks Generalize to DT Tasks?cited by 11
  4. 2025MetaSlot: Break Through the Fixed Number of Slots in Object-Centric Learningcited by 11
  5. 2024On the Ability of Deep Networks to Learn Symmetries from Data: A Neural Kernel Theorycited by 10
  6. 2023Visual Data-Type Understanding does not emerge from Scaling Vision-Language Modelscited by 9
  7. 2025When and How Does CLIP Enable Domain and Compositional Generalization?cited by 9
  8. 2024Skews in the Phenomenon Space Hinder Generalization in Text-to-Image Generationcited by 8
  9. 2025CoInD: Enabling Logical Compositions in Diffusion Modelscited by 8
  10. 2025Scaling can lead to compositional generalizationcited by 7
  11. 2025Circuit Stability Characterizes Language Model Generalizationcited by 6
  12. 2025SUB: Benchmarking CBM Generalization via Synthetic Attribute Substitutionscited by 6
  13. 2025Adversarial D'ej`a Vu: Jailbreak Dictionary Learning for Stronger Generalization to Unseen Attackscited by 6
  14. 2024What makes Models Compositional? A Theoretical View: With Supplementcited by 5
  15. 2024Swing-by Dynamics in Concept Learning and Compositional Generalizationcited by 5
  16. 2025Position: We Need An Algorithmic Understanding of Generative AIcited by 5
  17. 2025From Isolation to Entanglement: When Do Interpretability Methods Identify and Disentangle Known Concepts?cited by 5
  18. 2024A General Theory for Compositional Generalizationcited by 4
  19. 2024Unsupervised Composable Representations for Audiocited by 4
  20. 2025Towards Understanding Extrapolation: a Causal Lenscited by 4
  21. 2025Compositional Generalization via Forced Rendering of Disentangled Latentscited by 4
  22. 2025Compositional Scene Understanding through Inverse Generative Modelingcited by 3
  23. 2025What Drives Compositional Generalization in Visual Generative Models? The Importance of Continuous Training Objectivescited by 3
  24. 2026Unsupervised Decomposition and Recombination with Discriminator-Driven Diffusion Modelscited by 3
  25. 2024Compositional Conservatism: A Transductive Approach in Offline Reinforcement Learningcited by 2
  26. 2024Analyzing Generative Models by Manifold Entropic Metricscited by 2
  27. 2024Successes and Limitations of Object-centric Models at Compositional Generalisationcited by 2
  28. 2025Shedding Light in Task Decomposition in Program Synthesis: The Driving Force of the Synthesizer Modelcited by 2
  29. 2025Imbalance in Balance: Online Concept Balancing in Generation Modelscited by 2
  30. 2025State Combinatorial Generalization In Decision Making With Conditional Diffusion Modelscited by 1
  31. 2025Beyond Either-Or Reasoning: Transduction and Induction as Cooperative Problem-Solving Paradigmscited by 1
  32. 2025Separating the what and how of compositional computation to enable reuse and continual learningcited by 1
  33. 2025Learning by Analogy: A Causal Framework for Composition Generalizationcited by 1
  34. 2026Compositional Transduction with Latent Analogies for Offline Goal-Conditioned Reinforcement Learningcited by 1
  35. 2023Fast and Expressive Gesture Recognition using a Combination-Homomorphic Electromyogram Encodercited by 0
  36. 2024Compositional Models for Estimating Causal Effectscited by 0
  37. 2025Compositional World Knowledge leads to High Utility Synthetic datacited by 0
  38. 2025Extrapolation Guarantees for Perturbation Modeling Under the Additive Latent Shift Assumptioncited by 0

50 most cited of 62 All 62 (JSON)