Modularity is the Bedrock of Natural and Artificial Intelligence
Not on the map. Why?
Abstract (arXiv)
The remarkable performance of modern AI systems has been driven by unprecedented scales of data, computation, and energy -- far exceeding the resources required by human intelligence. This disparity highlights the need for new guiding principles and motivates drawing inspiration from the fundamental organizational principles of brain computation. Among these principles, modularity has been shown to be critical for supporting the efficient learning and strong generalization abilities consistently exhibited by humans. Furthermore, modularity aligns well with the No Free Lunch Theorem, which highlights the need for problem-specific inductive biases and motivates architectures composed of specialized components that solve subproblems. However, despite its fundamental role in natural intelligence and its demonstrated benefits across a range of seemingly disparate AI subfields, modularity remains relatively underappreciated in mainstream AI research. In this work, we review several research threads in artificial intelligence and neuroscience through a conceptual framework that highlights the central role of modularity in supporting both artificial and natural intelligence. In particular, we examine what computational advantages modularity provides, how it has emerged as a solution across several AI research areas, which modularity principles the brain exploits, and how modularity can help bridge the gap between natural and artificial intelligence.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 99 of its 215 references that are in the map. First posted on arXiv 21 Feb 2026.
Builds on
99 of 215 references.
References · 99
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023GPT-4 Technical Reportcited by 18,132
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2023Visual Instruction Tuningcited by 7,792
- 2017Mask R-CNNcited by 7,259
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
38 more
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
- 2013Visualizing and Understanding Convolutional Networkscited by 2,278
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
- 2020Shortcut Learning in Deep Neural Networkscited by 1,928
- 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
- 2017Gradient Episodic Memory for Continual Learningcited by 1,871
- 2024Mixtral of Expertscited by 1,785
- 2016Progressive Neural Networkscited by 1,332
- 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
- 2021Improving language models by retrieving from trillions of tokenscited by 1,278
- 2017Continual Learning with Deep Generative Replaycited by 1,152
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
- 2018Experience Replay for Continual Learningcited by 1,074
- 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
- 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
- 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
- 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
- 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
- 2024KAN: Kolmogorov-Arnold Networkscited by 634
- 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
- 2021Knowledge Neurons in Pretrained Transformerscited by 590
- 2023Textbooks Are All You Needcited by 561
- 2017Learning multiple visual domains with residual adapterscited by 553
- 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
- 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
- 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 425
- 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 422
- 2023Cognitive Architectures for Language Agentscited by 422
- 2023Augmented Language Models: a Surveycited by 398
- 2015Neural Module Networkscited by 375
- 2019The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervisioncited by 369
- 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 362
- 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 308
50 most cited of 99 All 99 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.