Data to 14 Oct 2026

Inductive Biases for Deep Learning of Higher-Level Cognition

arXiv 2011.15091 ↗First posted 30 Nov 2020cs.LG cs.AI stat.MLcited by 275

Not on the map. Why?

Abstract (arXiv)

A fascinating hypothesis is that human and animal intelligence could be explained by a few principles (rather than an encyclopedic list of heuristics). If that hypothesis was correct, we could more easily both understand our own intelligence and build intelligent machines. Just like in physics, the principles themselves would not be sufficient to predict the behavior of complex systems like brains, and substantial computation might be needed to simulate human-like intelligence. This hypothesis would suggest that studying the kind of inductive biases that humans and animals exploit could help both clarify these principles and provide inspiration for AI research and neuroscience theories. Deep learning already exploits several key inductive biases, and this work considers a larger list, focusing on those which concern mostly higher-level and sequential conscious processing. The objective of clarifying these particular principles is that they could potentially help us build AI systems benefiting from humans' abilities in terms of flexible out-of-distribution and systematic generalization, which is currently an area where a large gap exists between state-of-the-art machine learning and human intelligence.

Key facts. Cited by 275 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 41; 2022: 37; 2023: 64; 2024: 53; 2025: 45; 2026: 35.

275 papers.

Sort:

By area:

2023Large Language Modelscited by 780
2021GFlowNet Foundationscited by 290
2021D'ya like DAGs? A Survey on Structure Learning and Causal Discoverycited by 211
2023Generalization in diffusion models arises from geometry-adaptive harmonic representationscited by 185
2021Pretrained Transformers as Universal Computation Enginescited by 162
2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 160
2022Interventional Causal Representation Learningcited by 142
2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
2023Consciousness in Artificial Intelligence: Insights from the Science of Consciousnesscited by 104
2023How Does Information Bottleneck Help Deep Learning?cited by 93
2021Coordination Among Neural Modules Through a Shared Global Workspacecited by 85
2021Delving Deep into the Generalization of Vision Transformers under Distribution Shiftscited by 85

38 more

2021Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoningcited by 85
2024Robust agents learn causal world modelscited by 81
2021Neural Production Systems: Learning Rule-Governed Visual Dynamicscited by 75
2021Causal Influence Detection for Improving Efficiency in Reinforcement Learningcited by 71
2021A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challengescited by 69
2022DearKD: Data-Efficient Early Knowledge Distillation for Vision Transformerscited by 65
2021Visual Representation Learning Does Not Generalize Strongly Within the Same Domaincited by 61
2023Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolationcited by 58
2021Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learningcited by 53
2023Large Language Models Are Not Strong Abstract Reasonerscited by 53
2022A Unified Transformer Framework for Group-based Segmentation: Co-Segmentation, Co-Saliency Detection and Video Salient Object Detectioncited by 51
2025How Linguistics Learned to Stop Worrying and Love the Language Modelscited by 51
2025DiP: Taming Diffusion Models in Pixel Spacecited by 48
2021H-Transformer-1D: Fast One-Dimensional Hierarchical Attention for Sequencescited by 47
2024What to align in multimodal contrastive learning?cited by 47
2023Merlin:Empowering Multimodal LLMs with Foresight Mindscited by 46
2023Provably Learning Object-Centric Representationscited by 45
2022Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learningcited by 44
2024Towards a Mechanistic Explanation of Diffusion Model Generalizationcited by 42
2023Provable Compositional Generalization for Object-Centric Learningcited by 41
2023GFlowNet-EM for learning compositional latent variable modelscited by 40
2021Discrete-Valued Neural Communicationcited by 39
2021Learning General Optimal Policies with Graph Neural Networks: Expressive Power, Transparency, and Limitscited by 38
2021Fast and Slow Learning of Recurrent Independent Mechanismscited by 36
2022Is a Modular Architecture Enough?cited by 36
2021Trivial or impossible -- dichotomous data difficulty masks model differences (on ImageNet and beyond)cited by 35
2021From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligencecited by 35
2021Sparse Interventions in Language Models with Differentiable Maskingcited by 34
2022Dataset Condensation with Latent Space Knowledge Factorization and Sharingcited by 34
2021End-to-end Prostate Cancer Detection in bpMRI via 3D CNNs: Effects of Attention Mechanisms, Clinical Priori and Decoupled False Positive Reductioncited by 32
2022Partial Disentanglement via Mechanism Sparsitycited by 29
2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 28
2021A Consciousness-Inspired Planning Agent for Model-Based Reinforcement Learningcited by 27
2022EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithmcited by 27
2023Survey of Social Bias in Vision-Language Modelscited by 26
2021Dynamic Inference with Neural Interpreterscited by 24
2022Unifying Generative Models with GFlowNets and Beyondcited by 24
2024A Survey on Human Preference Learning for Large Language Modelscited by 24

50 most cited of 275 All 275 (JSON)