mlsanityData to 7 Oct 2026

Recurrent Independent Mechanisms

arXiv 1909.10893 ↗First posted 24 Sep 2019cs.LG cs.AI stat.MLcited by 219

Not on the map. Why?

Abstract (arXiv)

Learning modular structures which reflect the dynamics of the environment can lead to better generalization and robustness to changes which only affect a few of the underlying causes. We propose Recurrent Independent Mechanisms (RIMs), a new recurrent architecture in which multiple groups of recurrent cells operate with nearly independent transition dynamics, communicate only sparingly through the bottleneck of attention, and are only updated at time steps where they are most relevant. We show that this leads to specialization amongst the RIMs, which in turn allows for dramatically improved generalization on tasks where some factors of variation differ systematically between training and evaluation.

Key facts. Cited by 219 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 56; 2022: 63; 2023: 42; 2024: 29; 2025: 15; 2026: 13.

219 papers.

Sort:

By area:

  1. 2021GFlowNet Foundationscited by 286
  2. 2021Conditional Object-Centric Learning from Videocited by 247
  3. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 219
  4. 2021D'ya like DAGs? A Survey on Structure Learning and Causal Discoverycited by 209
  5. 2021Towards Causal Representation Learningcited by 207
  6. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  7. 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 158
  8. 2021Illiterate DALL-E Learns to Composecited by 147
  9. 2022SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Modelscited by 146
  10. 2021Inductive Biases and Variable Creation in Self-Attention Mechanismscited by 144
  11. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  12. 2021Generative Adversarial Transformerscited by 120
38 more
  1. 2021Independent mechanism analysis, a new concept?cited by 96
  2. 2021Coordination Among Neural Modules Through a Shared Global Workspacecited by 83
  3. 2023Modular Deep Learningcited by 82
  4. 2021Generalization and Robustness Implications in Object-Centric Learningcited by 81
  5. 2022Object discovery and representation networkscited by 80
  6. 2021Neural Production Systems: Learning Rule-Governed Visual Dynamicscited by 75
  7. 2021Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?cited by 75
  8. 2021Is Attention Better Than Matrix Decomposition?cited by 75
  9. 2021A Practical Survey on Faster and Lighter Transformerscited by 73
  10. 2020Fighting Copycat Agents in Behavioral Cloning from Observation Historiescited by 70
  11. 2021Causal Influence Detection for Improving Efficiency in Reinforcement Learningcited by 70
  12. 2022Self-Supervised Visual Representation Learning with Semantic Groupingcited by 70
  13. 2022When Physics Meets Machine Learning: A Survey of Physics-Informed Machine Learningcited by 64
  14. 2022Weakly Supervised Representation Learning with Sparse Perturbationscited by 64
  15. 2022Causal Dynamics Learning for Task-Independent State Abstractioncited by 64
  16. 2021Continual Learning via Local Module Compositioncited by 61
  17. 2022Improving Object-centric Learning with Query Optimizationcited by 60
  18. 2022Transferability in Deep Learning: A Surveycited by 57
  19. 2021CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fieldscited by 55
  20. 2021Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learningcited by 53
  21. 2022Retrieval-Augmented Reinforcement Learningcited by 53
  22. 2022Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Modelscited by 53
  23. 2022Assaying Out-Of-Distribution Generalization in Transfer Learningcited by 53
  24. 2021Attention, please! A survey of Neural Attention Models in Deep Learningcited by 50
  25. 2023An Investigation into Pre-Training Object-Centric Representations for Reinforcement Learningcited by 49
  26. 2022A survey of multimodal deep generative modelscited by 47
  27. 2022Dichotomy of Control: Separating What You Can Control from What You Cannotcited by 47
  28. 2022MoCoDA: Model-based Counterfactual Data Augmentationcited by 44
  29. 2023Model Agnostic Sample Reweighting for Out-of-Distribution Learningcited by 43
  30. 2022Causal de Finetti: On the Identification of Invariant Causal Structure in Exchangeable Datacited by 42
  31. 2022From Statistical to Causal Learningcited by 40
  32. 2022Complex-Valued Autoencoders for Object Discoverycited by 40
  33. 2021Discrete-Valued Neural Communicationcited by 39
  34. 2022History Compression via Language Models in Reinforcement Learningcited by 39
  35. 2022Modular Lifelong Reinforcement Learning via Neural Compositioncited by 39
  36. 2022The Neural Race Reduction: Dynamics of Abstraction in Gated Networkscited by 37
  37. 2022Neural Systematic Bindercited by 37
  38. 2023Emergent Modularity in Pre-trained Transformerscited by 37

50 most cited of 219 All 219 (JSON)