mlsanityData to 14 Oct 2026

Benchmarking Massively Parallelized Multi-Task Reinforcement Learning for Robotics Tasks

arXiv 2507.23172 ↗First posted 31 Jul 2025cs.ROcited by 11

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Multi-task Reinforcement Learning (MTRL) has emerged as a critical training paradigm for applying reinforcement learning (RL) to a set of complex real-world robotic tasks, which demands a generalizable and robust policy. At the same time, massively parallelized training has gained popularity, not only for significantly accelerating data collection through GPU-accelerated simulation but also for enabling diverse data collection across multiple tasks by simulating heterogeneous scenes in parallel. However, existing MTRL research has largely been limited to off-policy methods like SAC in the low-parallelization regime. MTRL could capitalize on the higher asymptotic performance of on-policy algorithms, whose batches require data from the current policy, and as a result, take advantage of massive parallelization offered by GPU-accelerated simulation. To bridge this gap, we introduce a massively parallelized Multi-Task Benchmark for robotics (MTBench), an open-sourced benchmark featuring a broad distribution of 50 manipulation tasks and 20 locomotion tasks, implemented using the GPU-accelerated simulator IsaacGym. MTBench also includes four base RL algorithms combined with seven state-of-the-art MTRL algorithms and architectures, providing a unified framework for evaluating their performance. Our extensive experiments highlight the superior speed of evaluating MTRL approaches using MTBench, while also uncovering unique challenges that arise from combining massive parallelism with MTRL. Code is available at https://github.com/Viraj-Joshi/MTBench

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 57 of its 71 references that are in the map. First posted on arXiv 31 Jul 2025.

Builds on

57 of 71 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Stabilizing Reinforcement Learning in Differentiable Multiphysics Simulation16 Dec

Other references · 56

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2016Layer Normalizationcited by 5,669
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  6. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  7. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  8. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  9. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  10. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  11. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  12. 2018DeepMind Control Suitecited by 956
38 more
  1. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  2. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  3. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  4. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
  5. 2018Learning Dexterous In-Hand Manipulationcited by 687
  6. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  7. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  8. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
  9. 2019Learning agile and dynamic motor skills for legged robotscited by 504
  10. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 503
  11. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  12. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
  13. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  14. 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 295
  15. 2018Distributed Prioritized Experience Replaycited by 269
  16. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 238
  17. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 229
  18. 2023Perpetual Humanoid Control for Real-time Simulated Avatarscited by 218
  19. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 174
  20. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 170
  21. 2022Rapid Locomotion via Reinforcement Learningcited by 168
  22. 2023Extreme Parkour with Legged Robotscited by 164
  23. 2023Robot Parkour Learningcited by 149
  24. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 143
  25. 2018Multi-task Deep Reinforcement Learning with PopArtcited by 141
  26. 2022Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotioncited by 121
  27. 2024HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulationcited by 112
  28. 2020Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous Reinforcement Learningcited by 93
  29. 2018GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learningcited by 90
  30. 2015Massively Parallel Methods for Deep Reinforcement Learningcited by 82
  31. 2024Simplifying Deep Temporal Difference Learningcited by 78
  32. 2023POPGym: Benchmarking Partially Observable Reinforcement Learningcited by 62
  33. 2023Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAXcited by 58
  34. 2023Pgx: Hardware-Accelerated Parallel Game Simulators for Reinforcement Learningcited by 51
  35. 2023Multi-Task Reinforcement Learning with Mixture of Orthogonal Expertscited by 49
  36. 2016Reinforcement Learning through Asynchronous Advantage Actor-Critic on a GPUcited by 48
  37. 2022PaCo: Parameter-Compositional Multi-Task Reinforcement Learningcited by 46
  38. 2021Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFingercited by 38

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 6.

11 papers.

  1. 2025Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agentscited by 12
  2. 2025Learning Massively Multitask World Models for Continuous Controlcited by 12
  3. 2025Scalable Multi-Objective and Meta Reinforcement Learning via Gradient Estimationcited by 8
  4. 2025Learning Without Critics? Revisiting GRPO in Classical Reinforcement Learning Environmentscited by 6
  5. 2026SEAR: Sample Efficient Action Chunking Reinforcement Learningcited by 5
  6. 2025Rethinking the Role of Dynamic Sparse Training for Scalable Deep Reinforcement Learningcited by 1
  7. 2026TOPPO: Rethinking PPO for Multi-Task Reinforcement Learning with Critic Balancingcited by 0
  8. 2026Distributionally Robust Multi-Task Reinforcement Learning via Adaptive Task Samplingcited by 0
  9. 2026A GPU-Parallel Framework for Heterogeneous Multi-Task Reinforcement Learningcited by 0
  10. 2026Temporal Self-Imitation Learningcited by 0
  11. 2026Touch2Robot: Robot Touch in the Human Demonstration Loopcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026T3S: Improving Multi-Task Reinforcement Learning with Task-Specific Feature Selector and Scheduler
  2. 2025Accelerating Model-Based Reinforcement Learning with State-Space World Models
  3. 2025AnyTask: an Automated Task and Data Generation Framework for Advancing Sim-to-Real Policy Learning
  4. 2026A Balanced Data Diet: Addressing the Exploration Bottleneck in Mega-Scale RL for Robot Control
  5. 2024Efficient Multi-Task Reinforcement Learning via Task-Specific Action Correction
  6. 2024D5RL: Diverse Datasets for Data-Driven Deep Reinforcement Learning
  7. 2025Leveraging Temporally Extended Behavior Sharing for Multi-task Reinforcement Learning
  8. 2026TMRL: Diffusion Timestep-Modulated Pretraining Enables Exploration for Efficient Policy Finetuning