mlsanityData to 14 Oct 2026

Distributionally Robust Multi-Task Reinforcement Learning via Adaptive Task Sampling

arXiv 2605.14350 ↗First posted 14 May 2026cs.LGcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Multi-task reinforcement learning (MTRL) aims to train a single agent to efficiently optimize performance across multiple tasks simultaneously. However, jointly optimizing all tasks often yields imbalanced learning: agents quickly solve easy tasks but learn slowly on harder ones. While prior work primarily attributes this imbalance to conflicting task gradients and proposes gradient manipulation or specialized architectures to address it, we instead focus on a distinct and under-explored challenge: imbalanced data allocation. Standard MTRL allocates an equal number of environment interactions to each task, which over-allocates data to easy tasks that require relatively few interactions to solve and under-allocates data to hard tasks that require substantially more experience to solve. To address this challenge, we introduce Distributionally Robust Adaptive Task Sampling (DRATS), an algorithm that adaptively prioritizes sampling tasks furthest from being solved. We derive DRATS by formalizing MTRL as a feasibility problem from which we derive a minimax objective for minimizing the worst-case return gap, the difference between a desired target return and the agent's return on a task. In benchmarks like MetaWorld-MT10 and MT50, DRATS improves data efficiency and increases worst-task performance compared to existing task sampling algorithms.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 50 of its 63 references that are in the map. First posted on arXiv 14 May 2026.

Builds on

50 of 63 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Meta-World+: An Improved, Standardized, RL Benchmark16 May
  2. 2025Multi-Task Reinforcement Learning Enables Parameter Scaling7 Mar

Other references · 48

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2013Auto-Encoding Variational Bayescited by 4,053
  3. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  4. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,247
  5. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  6. 2017Hindsight Experience Replaycited by 1,136
  7. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
  8. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
  9. 2023Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruningcited by 415
  10. 2023DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretrainingcited by 399
  11. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  12. 2018Maximum a Posteriori Policy Optimisationcited by 328
36 more
  1. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  2. 2022The Primacy Bias in Deep Reinforcement Learningcited by 247
  3. 2022Multi-Task Learning as a Bargaining Gamecited by 215
  4. 2017Teacher-Student Curriculum Learningcited by 210
  5. 2020Prioritized Level Replaycited by 198
  6. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 192
  7. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 174
  8. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 174
  9. 2019Distributionally Robust Language Modelingcited by 143
  10. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 143
  11. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 134
  12. 2021Rapid Exploration for Open-World Navigation with Latent Goal Modelscited by 121
  13. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 119
  14. 2020Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutionscited by 106
  15. 2020C-Learning: Learning to Achieve Goals via Recursive Classificationcited by 103
  16. 2020Automatic Curriculum Learning through Value Disagreementcited by 87
  17. 2018CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learningcited by 79
  18. 2023FAMO: Fast Adaptive Multitask Optimizationcited by 78
  19. 2023Efficient Online Data Mixing For Language Model Pre-Trainingcited by 77
  20. 2013Sample Complexity of Multi-task Reinforcement Learningcited by 58
  21. 2019Exploration via Hindsight Goal Generationcited by 54
  22. 2023Multi-Task Reinforcement Learning with Mixture of Orthogonal Expertscited by 49
  23. 2022PaCo: Parameter-Compositional Multi-Task Reinforcement Learningcited by 46
  24. 2020Non-convex Min-Max Optimization: Applications, Challenges, and Recent Theoretical Advancescited by 38
  25. 2021Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraftcited by 35
  26. 2020Why resampling outperforms reweighting for correcting sampling bias with stochastic gradientscited by 34
  27. 2022Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptationcited by 28
  28. 2024Robust Multi-Task Learning with Excess Riskscited by 24
  29. 2024A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoalscited by 23
  30. 2021C-Planning: An Automatic Curriculum for Learning Goal-Reaching Taskscited by 17
  31. 2023Outcome-directed Reinforcement Learning by Uncertainty & Temporal Distance-Aware Curriculum Goal Generationcited by 15
  32. 2025Distributionally Robust Reinforcement Learning with Human Feedbackcited by 12
  33. 2026Group Distributionally Robust Optimization-Driven Reinforcement Learning for LLM Reasoningcited by 12
  34. 2025Benchmarking Massively Parallelized Multi-Task Reinforcement Learning for Robotics Taskscited by 11
  35. 2021Balancing Average and Worst-case Accuracy in Multitask Learningcited by 10
  36. 2025AutoMixAlign: Adaptive Data Mixing for Multi-Task Preference Optimization in LLMscited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environments
  2. 2025Coreset-Based Task Selection for Sample-Efficient Meta-Reinforcement Learning
  3. 2024Sample Efficient Myopic Exploration Through Multitask Reinforcement Learning with Diverse Tasks
  4. 2024Shared-unique Features and Task-aware Prioritized Sampling on Multi-task Reinforcement Learning
  5. 2026Provable Multi-Task Reinforcement Learning: A Representation Learning Framework with Low Rank Rewards
  6. 2026Probabilistic Performance Guarantees for Multi-Task Reinforcement Learning
  7. 2026Scalable Multi-Task Inverse Reinforcement Learning
  8. 2024The Power of Active Multi-Task Learning in Reinforcement Learning from Human Feedback