mlsanityData to 14 Oct 2026

Predictive Representations for Skill Transfer in Reinforcement Learning

arXiv 2604.07016 ↗First posted 8 Apr 2026cs.LGcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

A key challenge in scaling up Reinforcement Learning is generalizing learned behaviour. Without the ability to carry forward acquired knowledge an agent is doomed to learn each task from scratch. In this paper we develop a new formalism for transfer by virtue of state abstraction. Based on task-independent, compact observations (outcomes) of the environment, we introduce Outcome-Predictive State Representations (OPSRs), agent-centered and task-independent abstractions that are made up of predictions of outcomes. We show formally and empirically that they have the potential for optimal but limited transfer, then overcome this trade-off by introducing OPSR-based skills, i.e. abstract actions (based on options) that can be reused between tasks as a result of state abstraction. In a series of empirical studies, we learn OPSR-based skills from demonstrations and show how they speed up learning considerably in entirely new and unseen tasks without any pre-processing. We believe that the framework introduced in this work is a promising step towards transfer in RL in general, and towards transfer through combining state and action abstraction specifically.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 12 of its 102 references that are in the map. First posted on arXiv 8 Apr 2026.

Builds on

12 of 102 references.

References · 12

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  3. 2016The Option-Critic Architecturecited by 448
  4. 2016Successor Features for Transfer in Reinforcement Learningcited by 340
  5. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
  6. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 203
  7. 2017A Laplacian Framework for Option Discovery in Reinforcement Learningcited by 110
  8. 2018Near-Optimal Representation Learning for Hierarchical Reinforcement Learningcited by 110
  9. 2017Multi-Level Discovery of Deep Optionscited by 51
  10. 2018Relational inductive bias for physical construction in humans and machinescited by 44
  11. 2020Self-supervised Visual Reinforcement Learning with Object-centric Representationscited by 44
  12. 2015Temporal-Difference Networkscited by 13

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024When Do Skills Help Reinforcement Learning? A Theoretical Analysis of Temporal Abstractions
  2. 2023Foundations for Transfer in Reinforcement Learning: A Taxonomy of Knowledge Modalities
  3. 2024Bridging State and History Representations: Understanding Self-Predictive RL
  4. 2025Context-Sensitive Abstractions for Reinforcement Learning with Parameterized Actions
  5. 2026OPID: On-Policy Skill Distillation for Agentic Reinforcement Learning
  6. 2021Representation Matters: Offline Pretraining for Sequential Decision Making
  7. 2024Zero-Shot Reinforcement Learning via Function Encoders
  8. 2025Goal-Oriented Skill Abstraction for Offline Multi-Task Reinforcement Learning