mlsanityData to 7 Oct 2026

The Relational Bottleneck as an Inductive Bias for Efficient Abstraction

arXiv 2309.06629 ↗First posted 12 Sep 2023cs.AI cs.NEcited by 26

Not on the map. Why?

Abstract (arXiv)

A central challenge for cognitive science is to explain how abstract concepts are acquired from limited experience. This has often been framed in terms of a dichotomy between connectionist and symbolic cognitive models. Here, we highlight a recently emerging line of work that suggests a novel reconciliation of these approaches, by exploiting an inductive bias that we term the relational bottleneck. In that approach, neural networks are constrained via their architecture to focus on relations between perceptual inputs, rather than the attributes of individual inputs. We review a family of models that employ this approach to induce abstractions in a data-efficient manner, emphasizing their potential as candidate models for the acquisition of abstract concepts in the human mind and brain.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 40 of its 138 references that are in the map. First posted on arXiv 12 Sep 2023.

Builds on

40 of 138 references.

References · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  4. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  5. 2022Emergent Abilities of Large Language Modelscited by 2,483
  6. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  7. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  8. 2020Object-Centric Learning with Slot Attentioncited by 865
  9. 2014Neural Turing Machinescited by 630
  10. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
  11. 2019The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervisioncited by 369
  12. 2017A simple neural network module for relational reasoningcited by 345
28 more
  1. 2018Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understandingcited by 304
  2. 2022Emergent Analogical Reasoning in Large Language Modelscited by 239
  3. 2017Neural Episodic Controlcited by 148
  4. 2017Inferring and Executing Programs for Visual Reasoningcited by 130
  5. 2019Learning Neural Causal Models from Unknown Interventionscited by 113
  6. 2016Learning to Compose Neural Networks for Question Answeringcited by 85
  7. 2021Neural Production Systems: Learning Rule-Governed Visual Dynamicscited by 75
  8. 2020Learning Compositional Rules via Neural Program Synthesiscited by 74
  9. 2020A comparison of Vector Symbolic Architecturescited by 63
  10. 2019Compositional generalization in a deep seq2seq model by separating syntax and semanticscited by 56
  11. 2020Emergent Symbols through Binding in External Memorycited by 47
  12. 2021Discrete-Valued Neural Communicationcited by 39
  13. 2020Understanding Human Intelligence through Human Limitationscited by 33
  14. 2022On Neural Architecture Inductive Biases for Relational Taskscited by 26
  15. 2017One Model for the Learning of Languagecited by 21
  16. 2023Learning to reason over visual objectscited by 19
  17. 2023Systematic Visual Reasoning through Object-Centric Relational Abstractioncited by 19
  18. 2021Probabilistic Analogical Mapping with Semantic Relation Networkscited by 14
  19. 2022GAMR: A Guided Attention Model for (visual) Reasoningcited by 12
  20. 2023Abstractors and relational cross-attention: An inductive bias for explicit relational reasoning in Transformerscited by 12
  21. 2021Visual analogy: Deep learning versus compositional modelscited by 11
  22. 2021Enriching Transformers with Structured Tensor-Product Representations for Abstractive Summarizationcited by 11
  23. 2017Question-Answering with Grammatically-Interpretable Representationscited by 8
  24. 2022Zero-shot visual reasoning through probabilistic analogical mappingcited by 8
  25. 2020Deep Predictive Learning in Neocortex and Pulvinarcited by 5
  26. 2020A Memory-Augmented Neural Network Model of Abstract Rule Learningcited by 4
  27. 2021Modelling the development of counting with memory-augmented neural networkscited by 4
  28. 2021The Structure of Systematicity in the Braincited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 10; 2025: 9; 2026: 4.

26 papers.

Sort:

By area:

  1. 2025Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Modelscited by 31
  2. 2024What is the Visual Cognition Gap between Humans and Multimodal LLMs?cited by 30
  3. 2023When can transformers reason with abstract symbols?cited by 23
  4. 2024Evidence from counterfactual tasks supports emergent analogical reasoning in large language modelscited by 14
  5. 2024MLPs Learn In-Context on Regression and Classification Taskscited by 12
  6. 2025Visual symbolic mechanisms: Emergent symbol processing in vision language modelscited by 12
  7. 2024Slot Abstractors: Toward Scalable Abstract Visual Reasoningcited by 10
  8. 2024From Frege to chatGPT: Compositionality in language, cognition, and deep neural networkscited by 7
  9. 2025Whither symbols in the era of advanced neural networks?cited by 6
  10. 2024Disentangling and Integrating Relational and Sensory Information in Transformer Architecturescited by 5
  11. 2023Deep Neural Networks Can Learn Generalizable Same-Different Visual Relationscited by 4
  12. 2024Human-Like Geometric Abstraction in Large Pre-trained Neural Networkscited by 4
14 more
  1. 2025Unraveling the geometry of visual relational reasoningcited by 3
  2. 2023Learning Hierarchical Relational Representations through Relational Convolutionscited by 2
  3. 2025Learning richness modulates equality reasoning in neural networkscited by 2
  4. 2025Language as a Wave Phenomenon: Semantic Phase Locking and Interference in Neural Networkscited by 2
  5. 2024Visual Reasoning in Object-Centric Deep Neural Networks: A Comparative Cognition Approachcited by 1
  6. 2024A Relational Inductive Bias for Dimensional Abstraction in Neural Networkscited by 1
  7. 2024LARS-VSA: A Vector Symbolic Architecture For Learning with Abstract Rulescited by 1
  8. 2025Systematic Abductive Reasoning via Diverse Relation Representations in Vector-symbolic Architecturecited by 1
  9. 2025Convolutional Neural Networks Can (Meta-)Learn the Same-Different Relationcited by 0
  10. 2025A Vision Toward Energy-Efficient Domain-Specific Artificial Intelligence Models and Agentscited by 0
  11. 2026A mathematical theory of balancing relational generalization and memorizationcited by 0
  12. 2026Relational Attention for Data-Efficient Language Modelingcited by 0
  13. 2026Exactness at Inference: A Representational Criterion for Out-of-Distribution Generalizationcited by 0
  14. 2026Neural Structural Reasoner: A Brain-inspired Architecture for Reasoning over Structured Knowledgecited by 0