mlsanityData to 7 Oct 2026

Composing Task Knowledge with Modular Successor Feature Approximators

arXiv 2301.12305 ↗First posted 28 Jan 2023cs.LG cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Recently, the Successor Features and Generalized Policy Improvement (SF&GPI) framework has been proposed as a method for learning, composing, and transferring predictive knowledge and behavior. SF&GPI works by having an agent learn predictive representations (SFs) that can be combined for transfer to new tasks with GPI. However, to be effective this approach requires state features that are useful to predict, and these state-features are typically hand-designed. In this work, we present a novel neural network architecture, "Modular Successor Feature Approximators" (MSFA), where modules both discover what is useful to predict, and learn their own predictive representations. We show that MSFA is able to better generalize compared to baseline architectures for learning SFs and modular architectures

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 33 of its 51 references that are in the map. First posted on arXiv 28 Jan 2023.

Builds on

33 of 51 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  3. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  4. 2020Reinforcement Learning with Augmented Datacited by 452
  5. 2016Successor Features for Transfer in Reinforcement Learningcited by 331
  6. 2019Stabilizing Transformers for Reinforcement Learningcited by 262
  7. 2018BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learningcited by 238
  8. 2019Recurrent Independent Mechanismscited by 219
  9. 2020Acme: A Research Framework for Distributed Reinforcement Learningcited by 146
  10. 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 131
  11. 2016Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfercited by 124
  12. 2020One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Controlcited by 114
21 more
  1. 2018Count-Based Exploration with the Successor Representationcited by 112
  2. 2018Universal Successor Features Approximatorscited by 102
  3. 2019Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneckcited by 101
  4. 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 97
  5. 2021MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Researchcited by 96
  6. 2019Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvementcited by 88
  7. 2018Composable Deep Reinforcement Learning for Robotic Manipulationcited by 70
  8. 2018Relational recurrent neural networkscited by 65
  9. 2021The Option Keyboard: Combining Skills in Reinforcement Learningcited by 65
  10. 2016Deep Reinforcement Learning with Successor Features for Navigation across Similar Environmentscited by 50
  11. 2017Visual Semantic Planning using Deep Successor Representationscited by 45
  12. 2019Discovery of Useful Questions as Auxiliary Taskscited by 45
  13. 2021Fast and Slow Learning of Recurrent Independent Mechanismscited by 36
  14. 2018Successor Uncertainties: Exploration and Uncertainty in Temporal Difference Learningcited by 35
  15. 2021Procedural Generalization by Planning with Self-Supervised World Modelscited by 30
  16. 2020Meta Reinforcement Learning with Autonomous Inference of Subtask Dependenciescited by 25
  17. 2021PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learningcited by 23
  18. 2021A Deep Reinforcement Learning Approach to Marginalized Importance Sampling with the Successor Representationcited by 19
  19. 2020Reinforcement Learning for Sparse-Reward Object-Interaction Tasks in a First-person Simulated 3D Environmentcited by 8
  20. 2021Reinforcement Learning of Implicit and Explicit Control Flow in Instructionscited by 4
  21. 2022Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalizationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 2; 2026: 1.

7 papers.

  1. 2023Combining Behaviors with the Successor Features Keyboardcited by 12
  2. 2024Learning Successor Features the Simple Waycited by 10
  3. 2024Predictive representations: building blocks of intelligencecited by 4
  4. 2025Constructing an Optimal Behavior Basis for the Option Keyboardcited by 2
  5. 2023Modularity Trumps Invariance for Compositional Robustnesscited by 0
  6. 2025A Unified Theory of Compositionality, Modularity, and Interpretability in Markov Decision Processescited by 0
  7. 2026Task diversity produces systematic transfer but inhibits continual reinforcement learningcited by 0