mlsanityData to 7 Oct 2026

Behavioural vs. Representational Systematicity in End-to-End Models: An Opinionated Survey

arXiv 2506.04461 ↗First posted 4 Jun 2025cs.LG cs.AI cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

A core aspect of compositionality, systematicity is a desirable property in ML models as it enables strong generalization to novel contexts. This has led to numerous studies proposing benchmarks to assess systematic generalization, as well as models and training regimes designed to enhance it. Many of these efforts are framed as addressing the challenge posed by Fodor and Pylyshyn. However, while they argue for systematicity of representations, existing benchmarks and models primarily focus on the systematicity of behaviour. We emphasize the crucial nature of this distinction. Furthermore, building on Hadley's (1994) taxonomy of systematic generalization, we analyze the extent to which behavioural systematicity is tested by key benchmarks in the literature across language and vision. Finally, we highlight ways of assessing systematicity of representations in ML models as practiced in the field of mechanistic interpretability.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 45 of its 95 references that are in the map. First posted on arXiv 4 Jun 2025.

Builds on

45 of 95 references.

References · 45

  1. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  2. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  3. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  4. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  5. 2020Object-Centric Learning with Slot Attentioncited by 865
  6. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  7. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 523
  8. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
  9. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
  10. 2019The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervisioncited by 369
  11. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  12. 2019Compositionality decomposed: how do neural networks generalise?cited by 264
33 more
  1. 2020COGS: A Compositional Generalization Challenge Based on Semantic Interpretationcited by 260
  2. 2020On the Binding Problem in Artificial Neural Networkscited by 230
  3. 2019RAVEN: A Dataset for Relational and Analogical Visual rEasoNingcited by 225
  4. 2022CREPE: Can Vision-Language Foundation Models Reason Compositionally?cited by 177
  5. 2020Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?cited by 144
  6. 2021The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformerscited by 117
  7. 2024Evaluating the World Model Implicit in a Generative Modelcited by 107
  8. 2023How do Language Models Bind Entities in Context?cited by 102
  9. 2022Does CLIP Bind Concepts? Probing Compositionality in Large Image Modelscited by 101
  10. 2023The ConceptARC Benchmark: Evaluating Understanding and Generalization in the ARC Domaincited by 97
  11. 2023Language Models Implement Simple Word2Vec-style Vector Arithmeticcited by 97
  12. 2023Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Taskcited by 91
  13. 2019Measuring Compositionality in Representation Learningcited by 84
  14. 2021Neural Production Systems: Learning Rule-Governed Visual Dynamicscited by 75
  15. 2021Visual Representation Learning Does Not Generalize Strongly Within the Same Domaincited by 60
  16. 2021The paradox of the compositionality of natural language: a neural machine translation case studycited by 58
  17. 2022A Benchmark for Compositional Visual Reasoningcited by 43
  18. 2024Monitoring Latent World States in Language Models with Propositional Probescited by 39
  19. 2021Neuro-Symbolic Inductive Logic Programming with Logical Neural Networkscited by 36
  20. 2018Jump to better conclusions: SCAN both left and rightcited by 29
  21. 2023ReCOGS: How Incidental Details of a Logical Form Overshadow an Evaluation of Semantic Interpretationcited by 24
  22. 2023SLOG: A Structural Generalization Benchmark for Semantic Parsingcited by 22
  23. 2022Systematic Generalization and Emergent Structures in Transformers Trained on Structured Taskscited by 19
  24. 2022Evaluating Understanding on Conceptual Abstraction Benchmarkscited by 13
  25. 2024Limits of Transformer Language Models on Learning to Compose Algorithmscited by 13
  26. 2023The Validity of Evaluation Results: Assessing Concurrence Across Compositionality Benchmarkscited by 12
  27. 2025Object-centric Binding in Contrastive Language-Image Pretrainingcited by 12
  28. 2019A Theory of Relation Learning and Cross-domain Generalizationcited by 11
  29. 2025Analogical Reasoning Inside Large Language Models: Concept Vectors and the Limits of Abstractioncited by 11
  30. 2024When can transformers compositionally generalize in-context?cited by 7
  31. 2023Imagine the Unseen World: A Benchmark for Systematic Generalization in Visual World Modelscited by 6
  32. 2024Compositional Generalization Across Distributional Shifts with Sparse Tree Operationscited by 5
  33. 2024Interpreting token compositionality in LLMs: A robustness analysiscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

  1. 2025Do Bias Benchmarks Generalise? Evidence from Voice-based Evaluation of Gender Bias in SpeechLLMscited by 7
  2. 2025How Do Language Models Compose Functions?cited by 7
  3. 2025DecompSR: A dataset for decomposed analyses of compositional multihop spatial reasoningcited by 1
  4. 2025Towards Integrated Alignmentcited by 0
  5. 2026Revisiting the Systematicity in Negation in the Era of In-Context Learningcited by 0