mlsanityData to 14 Oct 2026

Benchmarking Population-Based Reinforcement Learning across Robotic Tasks with GPU-Accelerated Simulation

arXiv 2404.03336 ↗First posted 4 Apr 2024cs.ROcited by 1

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

In recent years, deep reinforcement learning (RL) has shown its effectiveness in solving complex continuous control tasks. However, this comes at the cost of an enormous amount of experience required for training, exacerbated by the sensitivity of learning efficiency and the policy performance to hyperparameter selection, which often requires numerous trials of time-consuming experiments. This work leverages a Population-Based Reinforcement Learning (PBRL) approach and a GPU-accelerated physics simulator to enhance the exploration capabilities of RL by concurrently training multiple policies in parallel. The PBRL framework is benchmarked against three state-of-the-art RL algorithms -- PPO, SAC, and DDPG -- dynamically adjusting hyperparameters based on the performance of learning agents. The experiments are performed on four challenging tasks in Isaac Gym -- Anymal Terrain, Shadow Hand, Humanoid, Franka Nut Pick -- by analyzing the effect of population size and mutation mechanisms for hyperparameters. The results show that PBRL agents achieve superior performance, in terms of cumulative reward, compared to non-evolutionary baseline agents. Moreover, the trained agents are finally deployed in the real world for a Franka Nut Pick task. To our knowledge, this is the first sim-to-real attempt for deploying PBRL agents on real hardware. Code and videos of the learned policies are available on our project website (https://sites.google.com/view/pbrl).

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 27 of its 41 references that are in the map. First posted on arXiv 4 Apr 2024.

Builds on

27 of 41 references.

References · 27

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Mask R-CNNcited by 7,277
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2015Continuous control with deep reinforcement learningcited by 3,407
  5. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  6. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  7. 2019Solving Rubik's Cube with a Robot Handcited by 734
  8. 2018Learning Dexterous In-Hand Manipulationcited by 687
  9. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  10. 2019Learning agile and dynamic motor skills for legged robotscited by 504
  11. 2022Learning robust perceptive locomotion for quadrupedal robots in the wildcited by 382
  12. 2017Population Based Training of Neural Networkscited by 367
15 more
  1. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 264
  2. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 217
  3. 2022Rapid Locomotion via Reinforcement Learningcited by 168
  4. 2022DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Realitycited by 149
  5. 2022Accelerated Policy Learning with Parallel Differentiable Simulationcited by 104
  6. 2018GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learningcited by 90
  7. 2017Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigationcited by 77
  8. 2020Provably Efficient Online Hyperparameter Optimization with Population-Based Banditscited by 63
  9. 2022Factory: Fast Contact for Robotic Assemblycited by 54
  10. 2023IndustReal: Transferring Contact-Rich Assembly Tasks from Simulation to Realitycited by 53
  11. 2021Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFingercited by 38
  12. 2022Iterative Residual Policy: for Goal-Conditioned Dynamic Manipulation of Deformable Objectscited by 37
  13. 2023DexPBT: Scaling up Dexterous Manipulation for Hand-Arm Systems with Population Based Trainingcited by 29
  14. 2021Faster Improvement Rate Population Based Trainingcited by 13
  15. 2022Fast Population-Based Reinforcement Learning on a Single Machinecited by 7

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Simultaneous Training of First- and Second-Order Optimizers in Population-Based Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Generalized Population-Based Training for Hyperparameter Optimization in Reinforcement Learning
  2. 2024ARLBench: Flexible and Efficient Benchmarking for Hyperparameter Optimization in Reinforcement Learning
  3. 2024Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning
  4. 2025Gym4ReaL: A Suite for Benchmarking Real-World Reinforcement Learning
  5. 2025EvoRL: A GPU-accelerated Framework for Evolutionary Reinforcement Learning
  6. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes
  7. 2025Taxonomy and Trends in Reinforcement Learning for Robotics and Control Systems: A Structured Review
  8. 2025Accelerating Model-Based Reinforcement Learning with State-Space World Models