# Fast and Slow Learning of Recurrent Independent Mechanisms

arXiv: https://arxiv.org/abs/2105.08710 · HTML: https://mlsanity.com/paper/2105.08710/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 50 references that are in the map.
- First posted on arXiv 18 May 2021.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [RL2: Fast Reinforcement Learning via Slow Reinforcement Learning](https://mlsanity.com/paper/1611.02779/) (2016)
- [Learning to reinforcement learn](https://mlsanity.com/paper/1611.05763/) (2016)
- [Neural Module Networks](https://mlsanity.com/paper/1511.02799/) (2015)
- [PathNet: Evolution Channels Gradient Descent in Super Neural Networks](https://mlsanity.com/paper/1701.08734/) (2017)
- [Inductive Biases for Deep Learning of Higher-Level Cognition](https://mlsanity.com/paper/2011.15091/) (2020)
- [Recurrent Independent Mechanisms](https://mlsanity.com/paper/1909.10893/) (2019)
- [Meta-Gradient Reinforcement Learning](https://mlsanity.com/paper/1805.09801/) (2018)
- [A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms](https://mlsanity.com/paper/1901.10912/) (2019)
- [Inferring and Executing Programs for Visual Reasoning](https://mlsanity.com/paper/1705.03633/) (2017)
- [Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learning](https://mlsanity.com/paper/1711.01239/) (2017)
- [Learning Neural Causal Models from Unknown Interventions](https://mlsanity.com/paper/1910.01075/) (2019)
- [Building Generalizable Agents with a Realistic and Rich 3D Environment](https://mlsanity.com/paper/1801.02209/) (2018)
- [Neural Programmer-Interpreters](https://mlsanity.com/paper/1511.06279/) (2015)
- [Grounded Language Learning in a Simulated 3D World](https://mlsanity.com/paper/1706.06551/) (2017)
- [Evolved Policy Gradients](https://mlsanity.com/paper/1802.04821/) (2018)
- [AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence](https://mlsanity.com/paper/1905.10985/) (2019)
- [Coordination Among Neural Modules Through a Shared Global Workspace](https://mlsanity.com/paper/2103.01197/) (2021)
- [Discovering Reinforcement Learning Algorithms](https://mlsanity.com/paper/2007.08794/) (2020)
- [Gated-Attention Architectures for Task-Oriented Language Grounding](https://mlsanity.com/paper/1706.07230/) (2017)
- [Improving Generalization in Meta Reinforcement Learning using Learned Objectives](https://mlsanity.com/paper/1910.04098/) (2019)
- [Relational recurrent neural networks](https://mlsanity.com/paper/1806.01822/) (2018)
- [Modular Networks: Learning to Decompose Neural Computation](https://mlsanity.com/paper/1811.05249/) (2018)
- [Modular meta-learning](https://mlsanity.com/paper/1806.10166/) (2018)
- [Routing Networks and the Challenges of Modular and Compositional Computation](https://mlsanity.com/paper/1904.12774/) (2019)
- [Object Files and Schemata: Factorizing Declarative and Procedural Knowledge in Dynamical Systems](https://mlsanity.com/paper/2006.16225/) (2020)

## Built on by (most cited first, 30 of 36)

- [Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA](https://mlsanity.com/paper/2107.10098/) (2021)
- [Continual Learning via Local Module Composition](https://mlsanity.com/paper/2111.07736/) (2021)
- [Improving Object-centric Learning with Query Optimization](https://mlsanity.com/paper/2210.08990/) (2022)
- [MoCoDA: Model-based Counterfactual Data Augmentation](https://mlsanity.com/paper/2210.11287/) (2022)
- [Causal de Finetti: On the Identification of Invariant Causal Structure in Exchangeable Data](https://mlsanity.com/paper/2203.15756/) (2022)
- [Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI](https://mlsanity.com/paper/2111.06061/) (2021)
- [History Compression via Language Models in Reinforcement Learning](https://mlsanity.com/paper/2205.12258/) (2022)
- [Is a Modular Architecture Enough?](https://mlsanity.com/paper/2206.02713/) (2022)
- [Learning where to learn: Gradient sparsity in meta and continual learning](https://mlsanity.com/paper/2110.14402/) (2021)
- [Factored Adaptation for Non-Stationary Reinforcement Learning](https://mlsanity.com/paper/2203.16582/) (2022)
- [Sparsely Changing Latent States for Prediction and Planning in Partially Observable Domains](https://mlsanity.com/paper/2110.15949/) (2021)
- [Compositional Attention: Disentangling Search and Retrieval](https://mlsanity.com/paper/2110.09419/) (2021)
- [Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learning](https://mlsanity.com/paper/2210.07565/) (2022)
- [On the Generalization and Adaption Performance of Causal Models](https://mlsanity.com/paper/2206.04620/) (2022)
- [Composing Task Knowledge with Modular Successor Feature Approximators](https://mlsanity.com/paper/2301.12305/) (2023)
- [Unlocking the Potential of Generative AI through Neuro-Symbolic Architectures: Benefits and Limitations](https://mlsanity.com/paper/2502.11269/) (2025)
- [MC2-SF: Slow-Fast Learning for Mobile-Cloud Collaborative Recommendation](https://mlsanity.com/paper/2109.12314/) (2021)
- [Are All Vision Models Created Equal? A Study of the Open-Loop to Closed-Loop Causality Gap](https://mlsanity.com/paper/2210.04303/) (2022)
- [Breaking Neural Network Scaling Laws with Modularity](https://mlsanity.com/paper/2409.05780/) (2024)
- [Reusable Slotwise Mechanisms](https://mlsanity.com/paper/2302.10503/) (2023)
- [Dreamweaver: Learning Compositional World Models from Pixels](https://mlsanity.com/paper/2501.14174/) (2025)
- [A Fundamental Probabilistic Fuzzy Logic Framework Suitable for Causal Reasoning](https://mlsanity.com/paper/2205.15016/) (2022)
- [Learning to Decouple Complex Systems](https://mlsanity.com/paper/2302.01581/) (2023)
- [Socialized Learning: A Survey of the Paradigm Shift for Edge Intelligence in Networked Systems](https://mlsanity.com/paper/2404.13348/) (2024)
- [Attention-based Iterative Decomposition for Tensor Product Representation](https://mlsanity.com/paper/2406.01012/) (2024)
- [System 0/1/2/3: Quad-process theory for multi-timescale embodied collective cognitive systems](https://mlsanity.com/paper/2503.06138/) (2025)
- [The Thinking Pixel: Recursive Sparse Reasoning in Multimodal Diffusion Latents](https://mlsanity.com/paper/2604.25299/) (2026)
- [Hierarchical Reinforcement Learning for Temporal Abstraction of Listwise Recommendation](https://mlsanity.com/paper/2409.07416/) (2024)
- [PreND: Enhancing Intrinsic Motivation in Reinforcement Learning through Pre-trained Network Distillation](https://mlsanity.com/paper/2410.01745/) (2024)
- [Inductive Biases for Zero-shot Systematic Generalization in Language-informed Reinforcement Learning](https://mlsanity.com/paper/2501.15270/) (2025)

## Abstract (arXiv)

Decomposing knowledge into interchangeable pieces promises a generalization advantage when there are changes in distribution. A learning agent interacting with its environment is likely to be faced with situations requiring novel combinations of existing pieces of knowledge. We hypothesize that such a decomposition of knowledge is particularly relevant for being able to generalize in a systematic manner to out-of-distribution changes. To study these ideas, we propose a particular training framework in which we assume that the pieces of knowledge an agent needs and its reward function are stationary and can be re-used across tasks. An attention mechanism dynamically selects which modules can be adapted to the current task, and the parameters of the selected modules are allowed to change quickly as the learner is confronted with variations in what it experiences, while the parameters of the attention mechanisms act as stable, slowly changing, meta-parameters. We focus on pieces of knowledge captured by an ensemble of modules sparsely communicating with each other via a bottleneck of attention. We find that meta-learning the modular aspects of the proposed system greatly helps in achieving faster adaptation in a reinforcement learning setup involving navigation in a partially observed grid world with image-level input. We also find that reversing the role of parameters and meta-parameters does not work nearly as well, suggesting a particular role for fast adaptation of the dynamically selected modules.