mlsanityData to 7 Oct 2026

Offline Multitask Representation Learning for Reinforcement Learning

arXiv 2403.11574 ↗First posted 18 Mar 2024cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

We study offline multitask representation learning in reinforcement learning (RL), where a learner is provided with an offline dataset from different tasks that share a common representation and is asked to learn the shared representation. We theoretically investigate offline multitask low-rank RL, and propose a new algorithm called MORL for offline multitask representation learning. Furthermore, we examine downstream RL in reward-free, offline and online scenarios, where a new task is introduced to the agent that shares the same representation as the upstream offline tasks. Our theoretical results demonstrate the benefits of using the learned representation from the upstream offline task instead of directly learning the representation of the low-rank model.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 56 of its 94 references that are in the map. First posted on arXiv 18 Mar 2024.

Builds on

56 of 94 references.

References · 56

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  3. 2017Constrained Policy Optimizationcited by 928
  4. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  5. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  6. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 514
  7. 2020Is Pessimism Provably Efficient for Offline RL?cited by 362
  8. 2016Contextual Decision Processes with Low Bellman Rank are PAC-Learnablecited by 290
  9. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
  10. 2020FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPscited by 232
  11. 2017Distral: Robust Multitask Reinforcement Learningcited by 231
  12. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
38 more
  1. 2020Reward-Free Exploration for Reinforcement Learningcited by 177
  2. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  3. 2021Pessimistic Model-based Offline Reinforcement Learning under Partial Coveragecited by 160
  4. 2021Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learningcited by 157
  5. 2020What are the Statistical Limits of Offline RL with Linear Function Approximation?cited by 135
  6. 2021Representation Learning for Online and Offline RL in Low-rank MDPscited by 128
  7. 2022Pessimistic Bootstrapping for Uncertainty-Driven Offline Reinforcement Learningcited by 125
  8. 2021Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learningcited by 109
  9. 2020Fast active learning for pure exploration in reinforcement learningcited by 92
  10. 2020Offline Meta-Reinforcement Learning with Advantage Weightingcited by 92
  11. 2020PC-PG: Policy Cover Directed Exploration for Provable Policy Gradient Learningcited by 91
  12. 2020On Reward-Free Reinforcement Learning with Linear Function Approximationcited by 89
  13. 2021Model-free Representation Learning and Exploration in Low-rank MDPscited by 82
  14. 2021Towards Instance-Optimal Offline Reinforcement Learning with Pessimismcited by 78
  15. 2022Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trialscited by 78
  16. 2022Why So Pessimistic? Estimating Uncertainties for Offline RL through Ensembles, and Why Their Independence Matterscited by 74
  17. 2021Conservative Data Sharing for Multi-Task Offline Reinforcement Learningcited by 64
  18. 2022Near-optimal Offline Reinforcement Learning with Linear Representation: Leveraging Variance Information with Pessimismcited by 63
  19. 2022How to Leverage Unlabeled Data in Offline Reinforcement Learningcited by 61
  20. 2020Rewriting History with Inverse RL: Hindsight Inference for Policy Improvementcited by 59
  21. 2020Provably Efficient Reward-Agnostic Navigation with Linear Value Iterationcited by 56
  22. 2020Generalized Hindsight for Reinforcement Learningcited by 51
  23. 2022Efficient Reinforcement Learning in Block MDPs: A Model-free Representation Learning Approachcited by 51
  24. 2022Reward-Free RL is No Harder Than Reward-Aware RL in Linear Markov Decision Processescited by 49
  25. 2024Sharing Knowledge in Multi-Task Deep Reinforcement Learningcited by 42
  26. 2022Provable Benefits of Representational Transfer in Reinforcement Learningcited by 38
  27. 2020An Empirical Study of Representation Learning for Reinforcement Learning in Healthcarecited by 34
  28. 2021Lifelong Robotic Reinforcement Learning by Retaining Experiencescited by 34
  29. 2022Offline Reinforcement Learning Under Value and Density-Ratio Realizability: The Power of Gapscited by 34
  30. 2021On the Power of Multitask Representation Learning in Linear MDPcited by 30
  31. 2021Reward-Free Model-Based Reinforcement Learning with Linear Function Approximationcited by 29
  32. 2022Provable Benefit of Multitask Representation Learning in Reinforcement Learningcited by 29
  33. 2019SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewardscited by 28
  34. 2022On the Statistical Efficiency of Reward-Free Exploration in Non-Linear RLcited by 27
  35. 2020Near-Optimal Provable Uniform Convergence in Offline Policy Evaluation for Reinforcement Learningcited by 26
  36. 2022On the Feasibility of Cross-Task Transfer with Model-Based Reinforcement Learningcited by 15
  37. 2023Improved Sample Complexity for Reward-free Reinforcement Learning under Low-rank MDPscited by 14
  38. 2021Near-Optimal Reward-Free Exploration for Linear Mixture MDPs with Plug-in Solvercited by 13

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 6; 2026: 5.

14 papers.

Sort:

By area:

  1. 2024RLHF from Heterogeneous Feedback via Personalization and Preference Aggregationcited by 71
  2. 2024Contrastive Representation for Data Filtering in Cross-Domain Offline Reinforcement Learningcited by 18
  3. 2025A Survey of State Representation Learning for Deep Reinforcement Learningcited by 17
  4. 2025Learning Generalizable Skills from Offline Multi-Task Data for Multi-Agent Cooperationcited by 8
  5. 2025Task-Aware Virtual Training: Enhancing Generalization in Meta-Reinforcement Learning for Out-of-Distribution Taskscited by 3
  6. 2025Provable Zero-Shot Generalization in Offline Reinforcement Learningcited by 3
  7. 2025Pessimism Principle Can Be Effective: Towards a Framework for Zero-Shot Transfer Reinforcement Learningcited by 2
  8. 2024The Power of Active Multi-Task Learning in Reinforcement Learning from Human Feedbackcited by 1
  9. 2026VLGOR: Visual-Language Knowledge Guided Offline Reinforcement Learning for Generalizable Agentscited by 1
  10. 2026Provable Multi-Task Reinforcement Learning: A Representation Learning Framework with Low Rank Rewardscited by 1
  11. 2026IntentScore: Intent-Conditioned Action Evaluation for Computer-Use Agentscited by 1
  12. 2025Generalisation in Multitask Fitted Q-Iteration and Offline Q-learningcited by 0
2 more
  1. 2026Multi-Task Representation Learning for Conservative Linear Banditscited by 0
  2. 2026Robust Successor Featurescited by 0