mlsanityData to 7 Oct 2026

On the Binding Problem in Artificial Neural Networks

arXiv 2012.05208 ↗First posted 9 Dec 2020cs.NE cs.AI cs.LGcited by 230

Not on the map. Why?

Abstract (arXiv)

Contemporary neural networks still fall short of human-level generalization, which extends far beyond our direct experiences. In this paper, we argue that the underlying cause for this shortcoming is their inability to dynamically and flexibly bind information that is distributed throughout the network. This binding problem affects their capacity to acquire a compositional understanding of the world in terms of symbol-like entities (like objects), which is crucial for generalizing in predictable and systematic ways. To address this issue, we propose a unifying framework that revolves around forming meaningful entities from unstructured sensory inputs (segregation), maintaining this separation of information at a representational level (representation), and using these entities to construct new inferences, predictions, and behaviors (composition). Our analysis draws inspiration from a wealth of research in neuroscience and cognitive psychology, and surveys relevant mechanisms from the machine learning literature, to help identify a combination of inductive biases that allow symbolic information processing to emerge naturally in neural networks. We believe that a compositional approach to AI, in terms of grounded symbol-like representations, is of fundamental importance for realizing human-level generalization, and we hope that this paper may contribute towards that goal as a reference and inspiration.

Key facts. Cited by 230 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 32; 2022: 38; 2023: 43; 2024: 48; 2025: 37; 2026: 32.

230 papers.

Sort:

By area:

  1. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  2. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
  3. 2021Conditional Object-Centric Learning from Videocited by 247
  4. 2022Emergent Analogical Reasoning in Large Language Modelscited by 239
  5. 2022Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpaintingcited by 229
  6. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 219
  7. 2021D'ya like DAGs? A Survey on Structure Learning and Causal Discoverycited by 209
  8. 2021Towards Causal Representation Learningcited by 207
  9. 2022SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videoscited by 165
  10. 2021Illiterate DALL-E Learns to Composecited by 147
  11. 2022Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videoscited by 139
  12. 2022VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priorscited by 135
38 more
  1. 2021The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformerscited by 117
  2. 2021GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinementcited by 109
  3. 2023Consciousness in Artificial Intelligence: Insights from the Science of Consciousnesscited by 99
  4. 2022Object Scene Representation Transformercited by 90
  5. 2023Object-Centric Slot Diffusioncited by 90
  6. 2023SlotDiffusion: Object-Centric Generative Modeling with Diffusion Modelscited by 85
  7. 2023Modular Deep Learningcited by 82
  8. 2021Generalization and Robustness Implications in Object-Centric Learningcited by 81
  9. 2024Understanding the Limits of Vision Language Models Through the Lens of the Binding Problemcited by 71
  10. 2022Self-Supervised Visual Representation Learning with Semantic Groupingcited by 70
  11. 2021SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decompositioncited by 69
  12. 2021A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challengescited by 68
  13. 2021The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalizationcited by 62
  14. 2021Visual Representation Learning Does Not Generalize Strongly Within the Same Domaincited by 60
  15. 2022Improving Object-centric Learning with Query Optimizationcited by 60
  16. 2023Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolationcited by 56
  17. 2023Invariant Slot Attention: Object Discovery with Slot-Centric Reference Framescited by 51
  18. 2023An Investigation into Pre-Training Object-Centric Representations for Reinforcement Learningcited by 49
  19. 2023Self-supervised Object-Centric Learning for Videoscited by 49
  20. 2022Object Representations as Fixed Points: Training Iterative Refinement Algorithms with Implicit Differentiationcited by 46
  21. 2023Provably Learning Object-Centric Representationscited by 45
  22. 2021Computing on Functions Using Randomized Vector Representationscited by 41
  23. 2023Provable Compositional Generalization for Object-Centric Learningcited by 41
  24. 2022Complex-Valued Autoencoders for Object Discoverycited by 40
  25. 2022Neural Systematic Bindercited by 37
  26. 2021Efficient Iterative Amortized Inference for Learning Symmetric and Disentangled Multi-Object Representationscited by 36
  27. 2021Vector Symbolic Architectures as a Computing Framework for Emerging Hardwarecited by 35
  28. 2024Temporally Consistent Object-Centric Learning by Contrasting Slotscited by 30
  29. 2025CLIP Behaves like a Bag-of-Words Model Cross-modally but not Uni-modallycited by 30
  30. 2021Unsupervised Object Learning via Common Fatecited by 29
  31. 2023Rotating Features for Object Discoverycited by 29
  32. 2021Learning with Holographic Reduced Representationscited by 25
  33. 2022When are Lemons Purple? The Concept Association Bias of Vision-Language Modelscited by 23
  34. 2022Toward Compositional Generalization in Object-Oriented World Modelingcited by 22
  35. 2021Structured World Belief for Reinforcement Learning in POMDPcited by 21
  36. 2021Capturing the objects of vision with neural networkscited by 21
  37. 2022Learning What and Where: Disentangling Location and Identity Tracking Without Supervisioncited by 21
  38. 2023Slot-VAE: Object-Centric Scene Generation with Slot Attentioncited by 20

50 most cited of 230 All 230 (JSON)