mlsanityData to 14 Oct 2026

Feature-Attending Recurrent Modules for Generalization in Reinforcement Learning

arXiv 2112.08369 ↗First posted 15 Dec 2021cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Many important tasks are defined in terms of object. To generalize across these tasks, a reinforcement learning (RL) agent needs to exploit the structure that the objects induce. Prior work has either hard-coded object-centric features, used complex object-centric generative models, or updated state using local spatial features. However, these approaches have had limited success in enabling general RL agents. Motivated by this, we introduce "Feature-Attending Recurrent Modules" (FARM), an architecture for learning state representations that relies on simple, broadly applicable inductive biases for capturing spatial and temporal regularities. FARM learns a state representation that is distributed across multiple modules that each attend to spatiotemporal features with an expressive feature attention mechanism. We show that this improves an RL agent's ability to generalize across object-centric tasks. We study task suites in both 2D and 3D environments and find that FARM better generalizes compared to competing architectures that leverage attention or multiple modules.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 44 references that are in the map. First posted on arXiv 15 Dec 2021.

Builds on

32 of 44 references.

References · 32

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2017Squeeze-and-Excitation Networkscited by 4,789
  4. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  5. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  6. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,436
  7. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  8. 2020Object-Centric Learning with Slot Attentioncited by 876
  9. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  10. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
  11. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 392
  12. 2015Neural Module Networkscited by 376
20 more
  1. 2019Recurrent Independent Mechanismscited by 219
  2. 2017DARLA: Improving Zero-Shot Transfer in Reinforcement Learningcited by 149
  3. 2018Assessing Generalization in Deep Reinforcement Learningcited by 133
  4. 2019Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learningcited by 104
  5. 2018Universal Successor Features Approximatorscited by 103
  6. 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 98
  7. 2019Towards Interpretable Reinforcement Learning Using Attention Augmented Agentscited by 82
  8. 2018Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generationcited by 81
  9. 2017Gated-Attention Architectures for Task-Oriented Language Groundingcited by 72
  10. 2021SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decompositioncited by 69
  11. 2017Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physicscited by 58
  12. 2019COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Explorationcited by 57
  13. 2019Environmental drivers of systematicity and generalization in a situated agentcited by 50
  14. 2020Object Files and Schemata: Factorizing Declarative and Procedural Knowledge in Dynamical Systemscited by 48
  15. 2018Hierarchical Reinforcement Learning for Zero-shot Generalization with Subtask Dependenciescited by 43
  16. 2021Towards mental time travel: a hierarchical memory for reinforcement learning agentscited by 40
  17. 2020Meta Reinforcement Learning with Autonomous Inference of Subtask Dependenciescited by 25
  18. 2018Composable Planning with Attributescited by 21
  19. 2020Reinforcement Learning for Sparse-Reward Object-Interaction Tasks in a First-person Simulated 3D Environmentcited by 8
  20. 2021Reinforcement Learning of Implicit and Explicit Control Flow in Instructionscited by 4

Built on by

Not cited yet.