mlsanityData to 14 Oct 2026

Representation Learning Enables Scalable Multitask Deep Reinforcement Learning

arXiv 2606.05555 ↗First posted 4 Jun 2026cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Scaling reinforcement learning (RL) to diverse multitask settings remains a central challenge. While recent advances in model-based RL achieve strong performance, they rely on planning and complex training pipelines, making it unclear which components are essential for scalability. We revisit this question and argue that the primary driver of scalable multitask RL is not model-based control, but representation learning. In particular, we show that combining predictive, model-based representations with high-capacity value function approximation is sufficient to achieve strong performance, even without planning. We evaluate a simple model-free algorithm, MR.Q, coupled with auxiliary predictive objectives into a scalable actor-critic architecture. This approach outperforms a recent world-model-based method and a range of deep RL baselines across a diverse suite of multitask continuous control tasks, while significantly reducing computational overhead and improving wall-clock efficiency. We observe consistent improvements with increased model capacity and show through ablations that predictive representation learning is critical for performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 81 of its 97 references that are in the map. First posted on arXiv 4 Jun 2026.

Builds on

81 of 97 references.

Closest ancestors · 19

Same direction, newest first

  1. 2026The Surprising Difficulty of Search in Model-Based Reinforcement Learning29 Jan
  2. 2025Learning Massively Multitask World Models for Continuous Control24 Nov
  3. 2025A Survey of State Representation Learning for Deep Reinforcement Learning20 Jun
  4. 2025Stable Gradients for Stable Learning at Scale in Deep Reinforcement Learning18 Jun
  5. 2025The Impact of On-Policy Parallelized Data Collection on Deep Reinforcement Learning Networks3 Jun
  6. 2025Mastering Massive Multi-Task Reinforcement Learning via Mixture-of-Expert Decision Transformer30 May
  7. 2025Measure gradients, not activations! Enhancing neuronal activity in deep reinforcement learning29 May
  8. 2025Hadamax Encoding: Elevating Performance in Model-Free Atari21 May
  9. 2025Towards General-Purpose Model-Free Reinforcement Learning27 Jan
  10. 2024Neuroplastic Expansion in Deep Reinforcement Learning10 Oct
  11. 2024Don't flatten, tokenize! Unlocking the key to SoftMoE's efficacy in deep RL2 Oct
  12. 2024Improving Deep Reinforcement Learning by Reducing the Chain Effect of Value and Policy Churn7 Sep
7 more
  1. 2024PWM: Policy Learning with Multi-Task World Models2 Jul
  2. 2024On the consistency of hyper-parameter selection in value-based deep reinforcement learning25 Jun
  3. 2024Stop Regressing: Training Value Functions via Classification for Scalable Deep RL6 Mar
  4. 2024In value-based deep reinforcement learning, a pruned network is a good network19 Feb
  5. 2024Mixtures of Experts Unlock Parameter Scaling for Deep RL13 Feb
  6. 2024Sharing Knowledge in Multi-Task Deep Reinforcement Learning17 Jan
  7. 2024Bridging State and History Representations: Understanding Self-Predictive RL17 Jan

Other references · 62

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  7. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  8. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  9. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  10. 2018World Modelscited by 1,513
  11. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  12. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
38 more
  1. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  2. 2020Mastering Atari with Discrete World Modelscited by 1,007
  3. 2018DeepMind Control Suitecited by 956
  4. 2022A Generalist Agentcited by 808
  5. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  6. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  7. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  8. 2019Solving Rubik's Cube with a Robot Handcited by 734
  9. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  10. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
  11. 2019Model-Based Reinforcement Learning for Ataricited by 502
  12. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  13. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
  14. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  15. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  16. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
  17. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 274
  18. 2022The Primacy Bias in Deep Reinforcement Learningcited by 247
  19. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 238
  20. 2017Distral: Robust Multitask Reinforcement Learningcited by 233
  21. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 229
  22. 2025Video models are zero-shot learners and reasonerscited by 220
  23. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 207
  24. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 174
  25. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 174
  26. 2022What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?cited by 150
  27. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 147
  28. 2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 138
  29. 2025Training Agents Inside of Scalable World Modelscited by 136
  30. 2020Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learningcited by 134
  31. 2023Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behaviorcited by 131
  32. 2018Model-Based Reinforcement Learning via Meta-Policy Optimizationcited by 101
  33. 2020Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning Researchcited by 78
  34. 2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 75
  35. 2021On the Importance of Hyperparameter Optimization for Model-based Reinforcement Learningcited by 59
  36. 2020Predictive Information Accelerates Learning in RLcited by 56
  37. 2024Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learningcited by 50
  38. 2021MICo: Improved representations via sampling-based state similarity for Markov decision processescited by 43

50 most cited of 62 All 62 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Provable Benefit of Multitask Representation Learning in Reinforcement Learning
  2. 2025Multi-Task Reinforcement Learning Enables Parameter Scaling
  3. 2026Temporal Logic Guided Universal Task Representations for Reinforcement Learning
  4. 2026Provable Multi-Task Reinforcement Learning: A Representation Learning Framework with Low Rank Rewards
  5. 2023Rethinking Model-based, Policy-based, and Value-based Reinforcement Learning via the Lens of Representation Complexity
  6. 2025Towards Understanding the Benefit of Multitask Representation Learning in Decision Process
  7. 2024Efficient Exploration in Deep Reinforcement Learning: A Novel Bayesian Actor-Critic Algorithm
  8. 2025The Art of Scaling Reinforcement Learning Compute for LLMs