mlsanityData to 14 Oct 2026

The Limits of Transfer Reinforcement Learning with Latent Low-rank Structure

arXiv 2410.21601 ↗First posted 28 Oct 2024cs.LGcited by 1

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Many reinforcement learning (RL) algorithms are too costly to use in practice due to the large sizes S, A of the problem's state and action space. To resolve this issue, we study transfer RL with latent low rank structure. We consider the problem of transferring a latent low rank representation when the source and target MDPs have transition kernels with Tucker rank (S , d, A ), (S , S , d), (d, S, A ), or (d , d , d ). In each setting, we introduce the transfer-ability coefficient α that measures the difficulty of representational transfer. Our algorithm learns latent representations in each source MDP and then exploits the linear structure to remove the dependence on S, A , or S A in the target MDP regret bound. We complement our positive results with information theoretic lower bounds that show our algorithms (excluding the (d, d, d) setting) are minimax-optimal with respect to α.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 22 of its 40 references that are in the map. First posted on arXiv 28 Oct 2024.

Builds on

22 of 40 references.

References · 22

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  2. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 520
  3. 2020FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPscited by 233
  4. 2019Reinforcement Learning in Feature Space: Matrix Bandit, Kernels, and Regret Boundcited by 201
  5. 2020Nearly Minimax Optimal Reinforcement Learning for Linear Mixture Markov Decision Processescited by 200
  6. 2020Reward-Free Exploration for Reinforcement Learningcited by 177
  7. 2021Representation Learning for Online and Offline RL in Low-rank MDPscited by 129
  8. 2021Model-free Representation Learning and Exploration in Low-rank MDPscited by 82
  9. 2019A Survey of Deep Reinforcement Learning in Video Gamescited by 73
  10. 2022Nearly Minimax Optimal Reinforcement Learning for Linear Markov Decision Processescited by 57
  11. 2022Efficient Reinforcement Learning in Block MDPs: A Model-free Representation Learning Approachcited by 51
  12. 2022Provable Benefits of Representational Transfer in Reinforcement Learningcited by 38
10 more
  1. 2020Task-agnostic Exploration in Reinforcement Learningcited by 37
  2. 2021On the Power of Multitask Representation Learning in Linear MDPcited by 30
  3. 2022Provable Benefit of Multitask Representation Learning in Reinforcement Learningcited by 29
  4. 2019Harnessing Structures for Value-Based Planning and Reinforcement Learningcited by 24
  5. 2021Sample-Efficient Reinforcement Learning for Linearly-Parameterized MDPs with a Generative Modelcited by 18
  6. 2022Tensor and Matrix Low-Rank Value-Function Approximation in Reinforcement Learningcited by 16
  7. 2023Improved Sample Complexity for Reward-free Reinforcement Learning under Low-rank MDPscited by 14
  8. 2023Spectral Entry-wise Matrix Estimation for Low-Rank Reinforcement Learningcited by 10
  9. 2024Low-Rank Bandits via Tight Two-to-Infinity Singular Subspace Recoverycited by 6
  10. 2023Matrix Estimation for Offline Reinforcement Learning with Low-Rank Structurecited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Pessimism Principle Can Be Effective: Towards a Framework for Zero-Shot Transfer Reinforcement Learningcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Shift Before You Learn: Enabling Low-Rank Representations in Reinforcement Learning
  2. 2025Transition Transfer Q-Learning for Composite Markov Decision Processes
  3. 2025Accelerating Multi-Task Temporal Difference Learning under Low-Rank Representation
  4. 2021Provably Efficient Representation Selection in Low-rank Markov Decision Processes: From Online to Offline RL
  5. 2026The Low-Rank Structure of VLA Reinforcement Learning
  6. 2024Offline Multi-task Transfer RL with Representational Penalization
  7. 2024Model-free Low-Rank Reinforcement Learning via Leveraged Entry-wise Matrix Estimation
  8. 2025TD-JEPA: Latent-predictive Representations for Zero-Shot Reinforcement Learning