mlsanityData to 7 Oct 2026

Successor Features for Transfer in Reinforcement Learning

arXiv 1606.05312 ↗First posted 16 Jun 2016cs.AIcited by 331

Not on the map. Why?

Cited by 30 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Transfer in reinforcement learning refers to the notion that generalization should occur not only within a task but also across tasks. We propose a transfer framework for the scenario where the reward function changes between tasks but the environment's dynamics remain the same. Our approach rests on two key ideas: "successor features", a value function representation that decouples the dynamics of the environment from the rewards, and "generalized policy improvement", a generalization of dynamic programming's policy improvement operation that considers a set of policies rather than a single one. Put together, the two ideas lead to an approach that integrates seamlessly within the reinforcement learning framework and allows the free exchange of information across tasks. The proposed method also provides performance guarantees for the transferred policy even before any learning has taken place. We derive two theorems that set our approach in firm theoretical ground and present experiments that show that it successfully promotes transfer in practice, significantly outperforming alternative methods in a sequence of navigation tasks and in the control of a simulated robotic arm.

Key facts. Cited by 331 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Jun 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 1; 2020: 2; 2021: 49; 2022: 52; 2023: 45; 2024: 50; 2025: 47; 2026: 84.

331 papers.

Sort:

By area:

  1. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 256
  2. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  3. 2021A Practical Guide to Multi-Objective Reinforcement Learning and Planningcited by 240
  4. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  5. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  6. 2021Learning One Representation to Optimize All Rewardscited by 110
  7. 2021APS: Active Pretraining with Successor Featurescited by 109
  8. 2021Reward (Mis)design for Autonomous Drivingcited by 102
  9. 2024Foundation Policies with Hilbert Representationscited by 89
  10. 2022Does Zero-Shot Reinforcement Learning Exist?cited by 79
  11. 2023Preference Transformer: Modeling Human Preferences using Transformers for RLcited by 77
  12. 2025Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Modelscited by 77
38 more
  1. 2020Safe Imitation Learning via Fast Bayesian Reward Inference from Preferencescited by 73
  2. 2022CIC: Contrastive Intrinsic Control for Unsupervised Skill Discoverycited by 66
  3. 2021The Option Keyboard: Combining Skills in Reinforcement Learningcited by 65
  4. 2023Reinforcement Learning from Passive Data via Latent Intentionscited by 64
  5. 2022Lipschitz-constrained Unsupervised Skill Discoverycited by 60
  6. 2024PAD: Personalized Alignment of LLMs at Decoding-Timecited by 58
  7. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  8. 2021On The Effect of Auxiliary Tasks on Representation Dynamicscited by 56
  9. 2021Scalar reward is not enough: A response to Silver, Singh, Precup and Sutton (2021)cited by 55
  10. 2022Models of human preference for learning reward functionscited by 54
  11. 2024Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Makingcited by 53
  12. 2021Offline Meta-Reinforcement Learning with Online Self-Supervisioncited by 52
  13. 2021Temporal Abstraction in Reinforcement Learning with the Successor Representationcited by 52
  14. 2024Bridging State and History Representations: Understanding Self-Predictive RLcited by 52
  15. 2022Making Linear MDPs Practical via Contrastive Representation Learningcited by 49
  16. 2023Sample-Efficient Multi-Objective Learning via Generalized Policy Improvement Prioritizationcited by 46
  17. 2021MICo: Improved representations via sampling-based state similarity for Markov decision processescited by 43
  18. 2021Same State, Different Task: Continual Reinforcement Learning without Interferencecited by 42
  19. 2021Replacing Rewards with Examples: Example-Based Policy Search via Recursive Classificationcited by 38
  20. 2021Interesting Object, Curious Agent: Learning Task-Agnostic Explorationcited by 38
  21. 2022Bisimulation Makes Analogies in Goal-Conditioned Reinforcement Learningcited by 38
  22. 2022Provable Benefits of Representational Transfer in Reinforcement Learningcited by 38
  23. 2023Learning Zero-Shot Cooperation with Humans, Assuming Humans Are Biasedcited by 37
  24. 2022Discovering Policies with DOMiNO: Diversity Optimization Maintaining Near Optimalitycited by 35
  25. 2022Spectral Decomposition Representation for Reinforcement Learningcited by 35
  26. 2023Contrastive Difference Predictive Codingcited by 35
  27. 2024Unsupervised Zero-Shot Reinforcement Learning via Functional Reward Encodingscited by 31
  28. 2022Provable Benefit of Multitask Representation Learning in Reinforcement Learningcited by 29
  29. 2018Machine Teaching for Inverse Reinforcement Learning: Algorithms and Applicationscited by 28
  30. 2023Future-conditioned Unsupervised Pretraining for Decision Transformercited by 26
  31. 2021Block Contextual MDPs for Continual Learningcited by 25
  32. 2022Investigating the Properties of Neural Network Representations in Reinforcement Learningcited by 24
  33. 2024Cross-Domain Policy Adaptation by Capturing Representation Mismatchcited by 24
  34. 2021PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learningcited by 23
  35. 2021Successor Feature Landmarks for Long-Horizon Goal-Conditioned Reinforcement Learningcited by 23
  36. 2023Maximum State Entropy Exploration using Predecessor and Successor Representationscited by 23
  37. 2025Bigger, Regularized, Categorical: High-Capacity Value Functions are Efficient Multi-Task Learnerscited by 23
  38. 2021Return-based Scaling: Yet Another Normalisation Trick for Deep RLcited by 22

50 most cited of 331 All 331 (JSON)