# Universal Successor Representations for Transfer Reinforcement Learning

arXiv: https://arxiv.org/abs/1804.03758 · HTML: https://mlsanity.com/paper/1804.03758/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Apr 2018.

## Built on by (most cited first, 10 of 10)

- [A Practical Guide to Multi-Objective Reinforcement Learning and Planning](https://mlsanity.com/paper/2103.09568/) (2021)
- [Learning Successor States and Goal-Dependent Values: A Mathematical Viewpoint](https://mlsanity.com/paper/2101.07123/) (2021)
- [What is Essential for Unseen Goal Generalization of Offline Goal-conditioned RL?](https://mlsanity.com/paper/2305.18882/) (2023)
- [Maximum State Entropy Exploration using Predecessor and Successor Representations](https://mlsanity.com/paper/2306.14808/) (2023)
- [Successor Feature Sets: Generalizing Successor Representations Across Policies](https://mlsanity.com/paper/2103.02650/) (2021)
- [A Bayesian Approach to Identifying Representational Errors](https://mlsanity.com/paper/2103.15171/) (2021)
- [Multi-Agent Reinforcement Learning via Adaptive Kalman Temporal Difference and Successor Representation](https://mlsanity.com/paper/2112.15156/) (2021)
- [AKF-SR: Adaptive Kalman Filtering-based Successor Representation](https://mlsanity.com/paper/2204.00049/) (2022)
- [Deep Generative Models for Decision-Making and Control](https://mlsanity.com/paper/2306.08810/) (2023)
- [Uncertainty-aware transfer across tasks using hybrid model-based successor feature reinforcement learning](https://mlsanity.com/paper/2310.10818/) (2023)

## Abstract (arXiv)

The objective of transfer reinforcement learning is to generalize from a set of previous tasks to unseen new tasks. In this work, we focus on the transfer scenario where the dynamics among tasks are the same, but their goals differ. Although general value function (Sutton et al., 2011) has been shown to be useful for knowledge transfer, learning a universal value function can be challenging in practice. To attack this, we propose (1) to use universal successor representations (USR) to represent the transferable knowledge and (2) a USR approximator (USRA) that can be trained by interacting with the environment. Our experiments show that USR can be effectively applied to new tasks, and the agent initialized by the trained USRA can achieve the goal considerably faster than random initialization.