# Transfer with Model Features in Reinforcement Learning

arXiv: https://arxiv.org/abs/1807.01736 · HTML: https://mlsanity.com/paper/1807.01736/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jul 2018.

## Built on by (most cited first, 2 of 2)

- [Reward-Predictive Clustering](https://mlsanity.com/paper/2211.03281/) (2022)
- [Smaller Abstract State Spaces Enable Cross-Scale Generalization in Reinforcement Learning](https://mlsanity.com/paper/2605.20272/) (2026)

## Abstract (arXiv)

A key question in Reinforcement Learning is which representation an agent can learn to efficiently reuse knowledge between different tasks. Recently the Successor Representation was shown to have empirical benefits for transferring knowledge between tasks with shared transition dynamics. This paper presents Model Features: a feature representation that clusters behaviourally equivalent states and that is equivalent to a Model-Reduction. Further, we present a Successor Feature model which shows that learning Successor Features is equivalent to learning a Model-Reduction. A novel optimization objective is developed and we provide bounds showing that minimizing this objective results in an increasingly improved approximation of a Model-Reduction. Further, we provide transfer experiments on randomly generated MDPs which vary in their transition and reward functions but approximately preserve behavioural equivalence between states. These results demonstrate that Model Features are suitable for transfer between tasks with varying transition and reward functions.