mlsanityData to 14 Oct 2026

Measuring Compositionality in Representation Learning

arXiv 1902.07181 ↗First posted 19 Feb 2019cs.LG cs.CL stat.MLcited by 84

Not on the map. Why?

Abstract (arXiv)

Many machine learning algorithms represent input data with vector embeddings or discrete codes. When inputs exhibit compositional structure (e.g. objects built from parts or procedures from subroutines), it is natural to ask whether this compositional structure is reflected in the the inputs' learned representations. While the assessment of compositionality in languages has received significant attention in linguistics and adjacent fields, the machine learning literature lacks general-purpose tools for producing graded measurements of compositional structure in more general (e.g. vector-valued) representation spaces. We describe a procedure for evaluating compositionality by measuring how well the true representation-producing model can be approximated by a model that explicitly composes a collection of inferred representational primitives. We use the procedure to provide formal and empirical characterizations of compositional structure in a variety of settings, exploring the relationship between compositionality and learning dynamics, human judgments, representational similarity, and generalization.

Key facts. Cited by 84 papers in the map (data through 14 Oct 2026). First posted on arXiv 19 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 17; 2022: 19; 2023: 14; 2024: 21; 2025: 9; 2026: 4.

84 papers.

Sort:

By area:

  1. 2024Benchmarking Complex Instruction-Following with Multiple Constraints Compositioncited by 152
  2. 2023Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Taskcited by 93
  3. 2023Towards Understanding and Improving GFlowNet Trainingcited by 75
  4. 2023Linear Spaces of Meanings: Compositional Structures in Vision-Language Modelscited by 48
  5. 2024TC-Bench: Benchmarking Temporal Compositionality in Text-to-Video and Image-to-Video Generationcited by 43
  6. 2021Emergent Communication of Generalizationscited by 42
  7. 2022Compositional Generalization in Unsupervised Compositional Representation Learning: A Study on Disentanglement and Emergent Languagecited by 42
  8. 2022Characterizing Intrinsic Compositionality in Transformers with Tree Projectionscited by 41
  9. 2022Neural Systematic Bindercited by 37
  10. 2022Do Vision-Language Pretrained Models Learn Composable Primitive Concepts?cited by 31
  11. 2021EXplainable Neural-Symbolic Learning (X-NeSyL) methodology to fuse deep learning representations with expert knowledge graphs: the MonuMAI cultural heritage use casecited by 30
  12. 2022Emergent Communication: Generalization and Overfitting in Lewis Gamescited by 28
38 more
  1. 2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
  2. 2022Unit Testing for Concepts in Neural Networkscited by 23
  3. 2024Relational Composition in Neural Networks: A Survey and Call to Actioncited by 23
  4. 2022Toward Compositional Generalization in Object-Oriented World Modelingcited by 22
  5. 2021Improving Compositional Generalization with Self-Training for Data-to-Text Generationcited by 19
  6. 2023Learning to Extrapolate: A Transductive Approachcited by 19
  7. 2021Don't Take It Literally: An Edit-Invariant Sequence Loss for Text Generationcited by 17
  8. 2024A Review of the Applications of Deep Learning-Based Emergent Communicationcited by 17
  9. 2024Geometric Signatures of Compositionality Across a Language Model's Lifetimecited by 16
  10. 2021Zero-shot Node Classification with Decomposed Graph Prototype Networkcited by 15
  11. 2024On The Specialization of Neural Modulescited by 14
  12. 2023Towards More Human-like AI Communication: A Review of Emergent Communication Researchcited by 13
  13. 2024A Complexity-Based Theory of Compositionalitycited by 13
  14. 2021Distinguishing rule- and exemplar-based generalization in learning systemscited by 12
  15. 2022Benchmarking Compositionality with Formal Languagescited by 12
  16. 2022Greybox XAI: a Neural-Symbolic learning framework to produce interpretable predictions for image classificationcited by 12
  17. 2021Expressivity of Emergent Language is a Trade-off between Contextual Complexity and Unpredictabilitycited by 11
  18. 2022Clustering units in neural networks: upstream vs downstream informationcited by 11
  19. 2023Lewis's Signaling Game as beta-VAE For Natural Word Lengths and Segmentscited by 11
  20. 2025Meta-Thinking in LLMs via Multi-Agent Reinforcement Learning: A Surveycited by 10
  21. 2022Are Representations Built from the Ground Up? An Empirical Examination of Local Composition in Language Modelscited by 9
  22. 2022Compositionality as Lexical Symmetrycited by 8
  23. 2022Defending Compositionality in Emergent Languagescited by 8
  24. 2022Transparency Helps Reveal When Language Models Learn Meaningcited by 8
  25. 2022Robust and Controllable Object-Centric Learning through Energy-based Modelscited by 7
  26. 2024From Frege to chatGPT: Compositionality in language, cognition, and deep neural networkscited by 7
  27. 2024Towards Compositionality in Concept Learningcited by 7
  28. 2021On the Interplay Between Fine-tuning and Composition in Transformerscited by 6
  29. 2021Grounded Graph Decoding Improves Compositional Generalization in Question Answeringcited by 6
  30. 2023Basic syntax from speech: Spontaneous concatenation in unsupervised deep neural networkscited by 6
  31. 2023The Role of Linguistic Priors in Measuring Compositional Generalization of Vision-Language Modelscited by 6
  32. 2025A Theory of Initialisation's Impact on Specialisationcited by 6
  33. 2024Language Evolution with Deep Learningcited by 5
  34. 2024What makes Models Compositional? A Theoretical View: With Supplementcited by 5
  35. 2024Understanding Simplicity Bias towards Compositional Mappings via Learning Dynamicscited by 5
  36. 2024Swing-by Dynamics in Concept Learning and Compositional Generalizationcited by 5
  37. 2024Quantifying artificial intelligence through algorithmic generalizationcited by 5
  38. 2025Behavioural vs. Representational Systematicity in End-to-End Models: An Opinionated Surveycited by 5

50 most cited of 84 All 84 (JSON)