# A New Representation of Successor Features for Transfer across Dissimilar Environments

arXiv: https://arxiv.org/abs/2107.08426 · HTML: https://mlsanity.com/paper/2107.08426/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 21 references that are in the map.
- First posted on arXiv 18 Jul 2021.

## Builds on (most cited first, 11 of 11)

- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Successor Features for Transfer in Reinforcement Learning](https://mlsanity.com/paper/1606.05312/) (2016)
- [Transfer Learning in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2009.07888/) (2020)
- [CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-driving](https://mlsanity.com/paper/1807.03776/) (2018)
- [Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvement](https://mlsanity.com/paper/1901.10964/) (2019)
- [Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning](https://mlsanity.com/paper/1703.02949/) (2017)
- [The Option Keyboard: Combining Skills in Reinforcement Learning](https://mlsanity.com/paper/2106.13105/) (2021)
- [Uniform Error Bounds for Gaussian Process Regression with Application to Safe Control](https://mlsanity.com/paper/1906.01376/) (2019)
- [Deep Reinforcement Learning with Successor Features for Navigation across Similar Environments](https://mlsanity.com/paper/1612.05533/) (2016)
- [Universal Successor Features for Transfer Reinforcement Learning](https://mlsanity.com/paper/2001.04025/) (2020)
- [VUSFA:Variational Universal Successor Features Approximator to Improve Transfer DRL for Target Driven Visual Navigation](https://mlsanity.com/paper/1908.06376/) (2019)

## Built on by (most cited first, 13 of 13)

- [Contextualize Me -- The Case for Context in Reinforcement Learning](https://mlsanity.com/paper/2202.04500/) (2022)
- [Maximum State Entropy Exploration using Predecessor and Successor Representations](https://mlsanity.com/paper/2306.14808/) (2023)
- [CARL: A Benchmark for Contextual and Adaptive Reinforcement Learning](https://mlsanity.com/paper/2110.02102/) (2021)
- [Combining Behaviors with the Successor Features Keyboard](https://mlsanity.com/paper/2310.15940/) (2023)
- [Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matching](https://mlsanity.com/paper/2411.07007/) (2024)
- [Semantic-Aware Collaborative Deep Reinforcement Learning Over Wireless Cellular Networks](https://mlsanity.com/paper/2111.12064/) (2021)
- [Predictive representations: building blocks of intelligence](https://mlsanity.com/paper/2402.06590/) (2024)
- [Which Features are Best for Successor Features?](https://mlsanity.com/paper/2502.10790/) (2025)
- [Efficient Bayesian Policy Reuse with a Scalable Observation Model in Deep Reinforcement Learning](https://mlsanity.com/paper/2204.07729/) (2022)
- [Safety-Constrained Policy Transfer with Successor Features](https://mlsanity.com/paper/2211.05361/) (2022)
- [Ensemble Successor Representations for Task Generalization in Offline-to-Online Reinforcement Learning](https://mlsanity.com/paper/2405.07223/) (2024)
- [Contextual Intelligence The Next Leap for Reinforcement Learning](https://mlsanity.com/paper/2604.02348/) (2026)
- [ConTraIRL: Factorized Contrastive Abstractions for Transferable IRL](https://mlsanity.com/paper/2606.03017/) (2026)

## Abstract (arXiv)

Transfer in reinforcement learning is usually achieved through generalisation across tasks. Whilst many studies have investigated transferring knowledge when the reward function changes, they have assumed that the dynamics of the environments remain consistent. Many real-world RL problems require transfer among environments with different dynamics. To address this problem, we propose an approach based on successor features in which we model successor feature functions with Gaussian Processes permitting the source successor features to be treated as noisy measurements of the target successor feature function. Our theoretical analysis proves the convergence of this approach as well as the bounded error on modelling successor feature functions with Gaussian Processes in environments with both different dynamics and rewards. We demonstrate our method on benchmark datasets and show that it outperforms current baselines.