mlsanityData to 7 Oct 2026

Fast and Slow Learning of Recurrent Independent Mechanisms

arXiv 2105.08710 ↗First posted 18 May 2021cs.LG cs.AIcited by 36

Not on the map. Why?

Abstract (arXiv)

Decomposing knowledge into interchangeable pieces promises a generalization advantage when there are changes in distribution. A learning agent interacting with its environment is likely to be faced with situations requiring novel combinations of existing pieces of knowledge. We hypothesize that such a decomposition of knowledge is particularly relevant for being able to generalize in a systematic manner to out-of-distribution changes. To study these ideas, we propose a particular training framework in which we assume that the pieces of knowledge an agent needs and its reward function are stationary and can be re-used across tasks. An attention mechanism dynamically selects which modules can be adapted to the current task, and the parameters of the selected modules are allowed to change quickly as the learner is confronted with variations in what it experiences, while the parameters of the attention mechanisms act as stable, slowly changing, meta-parameters. We focus on pieces of knowledge captured by an ensemble of modules sparsely communicating with each other via a bottleneck of attention. We find that meta-learning the modular aspects of the proposed system greatly helps in achieving faster adaptation in a reinforcement learning setup involving navigation in a partially observed grid world with image-level input. We also find that reversing the role of parameters and meta-parameters does not work nearly as well, suggesting a particular role for fast adaptation of the dynamically selected modules.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 35 of its 50 references that are in the map. First posted on arXiv 18 May 2021.

Builds on

35 of 50 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  6. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  7. 2015Neural Module Networkscited by 375
  8. 2016Learning to reinforcement learncited by 369
  9. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 362
  10. 2020Inductive Biases for Deep Learning of Higher-Level Cognitioncited by 273
  11. 2019Recurrent Independent Mechanismscited by 219
  12. 2019A Meta-Transfer Objective for Learning to Disentangle Causal Mechanismscited by 158
23 more
  1. 2018Meta-Gradient Reinforcement Learningcited by 156
  2. 2017Inferring and Executing Programs for Visual Reasoningcited by 130
  3. 2017Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learningcited by 125
  4. 2019Learning Neural Causal Models from Unknown Interventionscited by 113
  5. 2018Building Generalizable Agents with a Realistic and Rich 3D Environmentcited by 109
  6. 2017Grounded Language Learning in a Simulated 3D Worldcited by 100
  7. 2015Neural Programmer-Interpreterscited by 96
  8. 2018Evolved Policy Gradientscited by 86
  9. 2019AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligencecited by 84
  10. 2021Coordination Among Neural Modules Through a Shared Global Workspacecited by 83
  11. 2020Discovering Reinforcement Learning Algorithmscited by 75
  12. 2017Gated-Attention Architectures for Task-Oriented Language Groundingcited by 72
  13. 2019Improving Generalization in Meta Reinforcement Learning using Learned Objectivescited by 68
  14. 2018Relational recurrent neural networkscited by 65
  15. 2018Modular Networks: Learning to Decompose Neural Computationcited by 62
  16. 2018Modular meta-learningcited by 51
  17. 2019Routing Networks and the Challenges of Modular and Compositional Computationcited by 49
  18. 2020Object Files and Schemata: Factorizing Declarative and Procedural Knowledge in Dynamical Systemscited by 48
  19. 2020Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modulescited by 38
  20. 2018Sparse Attentive Backtracking: Temporal CreditAssignment Through Remindingcited by 33
  21. 2018Interactive Grounded Language Acquisition and Generalization in a 2D Worldcited by 21
  22. 2020S2RMs: Spatially Structured Recurrent Modulescited by 10
  23. 2018Modularity Matters: Learning Invariant Relational Reasoning Taskscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 12; 2023: 4; 2024: 6; 2025: 4; 2026: 3.

36 papers.

Sort:

By area:

  1. 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 158
  2. 2021Continual Learning via Local Module Compositioncited by 61
  3. 2022Improving Object-centric Learning with Query Optimizationcited by 60
  4. 2022MoCoDA: Model-based Counterfactual Data Augmentationcited by 44
  5. 2022Causal de Finetti: On the Identification of Invariant Causal Structure in Exchangeable Datacited by 42
  6. 2022History Compression via Language Models in Reinforcement Learningcited by 39
  7. 2021Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AIcited by 38
  8. 2022Is a Modular Architecture Enough?cited by 35
  9. 2021Learning where to learn: Gradient sparsity in meta and continual learningcited by 33
  10. 2022Factored Adaptation for Non-Stationary Reinforcement Learningcited by 30
  11. 2021Sparsely Changing Latent States for Prediction and Planning in Partially Observable Domainscited by 18
  12. 2021Compositional Attention: Disentangling Search and Retrievalcited by 12
24 more
  1. 2022Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learningcited by 11
  2. 2022On the Generalization and Adaption Performance of Causal Modelscited by 8
  3. 2023Composing Task Knowledge with Modular Successor Feature Approximatorscited by 7
  4. 2025Unlocking the Potential of Generative AI through Neuro-Symbolic Architectures: Benefits and Limitationscited by 6
  5. 2021MC2-SF: Slow-Fast Learning for Mobile-Cloud Collaborative Recommendationcited by 5
  6. 2022Are All Vision Models Created Equal? A Study of the Open-Loop to Closed-Loop Causality Gapcited by 5
  7. 2024Breaking Neural Network Scaling Laws with Modularitycited by 5
  8. 2023Reusable Slotwise Mechanismscited by 4
  9. 2025Dreamweaver: Learning Compositional World Models from Pixelscited by 4
  10. 2022A Fundamental Probabilistic Fuzzy Logic Framework Suitable for Causal Reasoningcited by 3
  11. 2023Learning to Decouple Complex Systemscited by 3
  12. 2024Socialized Learning: A Survey of the Paradigm Shift for Edge Intelligence in Networked Systemscited by 2
  13. 2024Attention-based Iterative Decomposition for Tensor Product Representationcited by 2
  14. 2025System 0/1/2/3: Quad-process theory for multi-timescale embodied collective cognitive systemscited by 2
  15. 2024Hierarchical Reinforcement Learning for Temporal Abstraction of Listwise Recommendationcited by 1
  16. 2024PreND: Enhancing Intrinsic Motivation in Reinforcement Learning through Pre-trained Network Distillationcited by 1
  17. 2026The Thinking Pixel: Recursive Sparse Reasoning in Multimodal Diffusion Latentscited by 1
  18. 2022Syntactic Inductive Biases for Deep Learning Methodscited by 0
  19. 2022Causal Coupled Mechanisms: A Control Method with Cooperation and Competition for Complex Systemcited by 0
  20. 2023RigLSTM: Recurrent Independent Grid LSTM for Generalizable Sequence Learningcited by 0
  21. 2024CAREL: Instruction-guided reinforcement learning with cross-modal auxiliary objectivescited by 0
  22. 2025Inductive Biases for Zero-shot Systematic Generalization in Language-informed Reinforcement Learningcited by 0
  23. 2026Differentiable Learning of Lifted Action Schemas for Classical Planningcited by 0
  24. 2026Position: Artificial Intelligence Needs Meta Intelligence -- the Case for Metacognitive AIcited by 0