mlsanityData to 14 Oct 2026

A GPU-Parallel Framework for Heterogeneous Multi-Task Reinforcement Learning

arXiv 2606.03335 ↗First posted 2 Jun 2026cs.ROcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

GPU-parallel simulation provides abundant robot interaction, but existing benchmarks rarely combine this scale with heterogeneous manipulation tasks and standardized multi-task RL evaluation. We introduce Hebero (Heterogeneous Benchmark for Robot Learning), a GPU-parallel Isaac Lab benchmark that enables efficient joint training and evaluation of a single policy across all 40 heterogeneous tasks. Scaling experiments show that increasing parallel replicas per task improves success under a fixed wall-clock budget. To support learning with sparse rewards and limited demonstrations, we propose Demonstration-Guided Policy Optimization (DGPO), which reuses demonstrations for dense tracking rewards and asymmetric value learning. Its shared stack supports controlled comparisons of learner-specific demonstration interfaces within PPO. Within DGPO framework, we introduce IW-ABC, which uses a lightweight per-task learning progress signal to coordinate adaptive behavior cloning (ABC), relaxing demonstration guidance with task progress, and importance weighting (IW), emphasizing lagging tasks in PPO updates. With 50 demonstrations per task, IW-ABC achieves 90.1% state-input mean success, outperforming the strongest baseline FAMO-ABC by 7.8 percentage points. Its visual counterpart reaches 93.5% mean success. Real-world experiments further demonstrate that a single multi-task policy trained in simulation can successfully perform four tasks on a physical Piper robot. The project page is available at https://hebero-rl.github.io/.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 30 of its 37 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

30 of 37 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interaction30 Sep
  2. 2025Benchmarking Massively Parallelized Multi-Task Reinforcement Learning for Robotics Tasks31 Jul
  3. 2025Goal-based Self-Adaptive Generative Adversarial Imitation Learning (Goal-SAGAIL) for Multi-goal Robotic Manipulation Tasks15 Jun
  4. 2025Multi-Stage Manipulation with Demonstration-Augmented Reward, Policy, and World Model Learning3 Mar
  5. 2024Multi Task Inverse Reinforcement Learning for Common Sense Reward17 Feb

Other references · 25

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  3. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  4. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  5. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
  6. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  7. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
  8. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
  9. 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 429
  10. 2023Efficient Online Reinforcement Learning with Offline Datacited by 363
  11. 2025RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twinscited by 121
  12. 2023FAMO: Fast Adaptive Multitask Optimizationcited by 78
13 more
  1. 2024Theia: Distilling Diverse Vision Foundation Models for Robot Learningcited by 61
  2. 2024Sharing Knowledge in Multi-Task Deep Reinforcement Learningcited by 44
  3. 2025Meta-World+: An Improved, Standardized, RL Benchmarkcited by 43
  4. 2025Learning Massively Multitask World Models for Continuous Controlcited by 12
  5. 2024DrS: Learning Reusable Dense Rewards for Multi-Stage Taskscited by 10
  6. 2025Scalable Multi-Objective and Meta Reinforcement Learning via Gradient Estimationcited by 8
  7. 2024Pessimistic Value Iteration for Multi-Task Data Sharing in Offline Reinforcement Learningcited by 7
  8. 2025RoboScape-R: Unified Reward-Observation World Models for Generalizable Robotics Training via RLcited by 5
  9. 2024Sample Efficient Myopic Exploration Through Multitask Reinforcement Learning with Diverse Taskscited by 2
  10. 2026MO-Playground: Massively Parallelized Multi-Objective Reinforcement Learning for Roboticscited by 2
  11. 2026Rainbow-DemoRL: Combining Improvements in Demonstration-Augmented Reinforcement Learningcited by 2
  12. 2024Learning Transparent Reward Models via Unsupervised Feature Selectioncited by 1
  13. 2025Data-Efficient Multitask DAggercited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025AnyTask: an Automated Task and Data Generation Framework for Advancing Sim-to-Real Policy Learning
  2. 2025Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning
  3. 2026Scaling Vision-Language Reward Learning for Robot Manipulation in Parallel Simulation
  4. 2024Active Fine-Tuning of Multi-Task Policies
  5. 2026X2Real: an eXtensive simulation benchmark for real-world generalist policies ↗
  6. 2024Going into Orbit: Massively Parallelizing Episodic Reinforcement Learning
  7. 2026MINERVA: How Small Can a Manipulation Policy Be and Still Solve LIBERO?
  8. 2025SpeedAug: Policy Acceleration via Tempo-Enriched Policy and RL Fine-Tuning