mlsanityData to 13 Oct 2026

High-Performance Reinforcement Learning on Spot: Optimizing Simulation Parameters with Distributional Measures

arXiv 2504.17857 ↗First posted 24 Apr 2025cs.LG cs.ROcited by 5

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

This work presents an overview of the technical details behind a high performance reinforcement learning policy deployment with the Spot RL Researcher Development Kit for low level motor access on Boston Dynamics Spot. This represents the first public demonstration of an end to end end reinforcement learning policy deployed on Spot hardware with training code publicly available through Nvidia IsaacLab and deployment code available through Boston Dynamics. We utilize Wasserstein Distance and Maximum Mean Discrepancy to quantify the distributional dissimilarity of data collected on hardware and in simulation to measure our sim2real gap. We use these measures as a scoring function for the Covariance Matrix Adaptation Evolution Strategy to optimize simulated parameters that are unknown or difficult to measure from Spot. Our procedure for modeling and training produces high quality reinforcement learning policies capable of multiple gaits, including a flight phase. We deploy policies capable of over 5.2ms locomotion, more than triple Spots default controller maximum speed, robustness to slippery surfaces, disturbance rejection, and overall agility previously unseen on Spot. We detail our method and release our code to support future work on Spot with the low level API.

Key facts. Cited by 5 papers in the map (data through 13 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 23 of its 49 references that are in the map. First posted on arXiv 24 Apr 2025.

Builds on

23 of 49 references.

References · 23

  1. 2017Proximal Policy Optimization Algorithmscited by 14,592
  2. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 434
  3. 2019Learning agile and dynamic motor skills for legged robotscited by 406
  4. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 399
  5. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 279
  6. 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 240
  7. 2018Pros and Cons of GAN Evaluation Measurescited by 224
  8. 2018Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networkscited by 171
  9. 2015Training generative neural networks via Maximum Mean Discrepancy optimizationcited by 149
  10. 2022Rapid Locomotion via Reinforcement Learningcited by 130
  11. 2023ANYmal Parkour: Learning Agile Navigation for Quadrupedal Robotscited by 107
  12. 2021Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robotscited by 95
11 more
  1. 2022Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotioncited by 61
  2. 2021Robot Learning from Randomized Simulations: A Reviewcited by 56
  3. 2020Sim2Real Transfer for Reinforcement Learning without Dynamics Randomizationcited by 43
  4. 2018Feedback Control For Cassie With Deep Reinforcement Learningcited by 36
  5. 2022Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrationscited by 30
  6. 2020Hardware as Policy: Mechanical and Computational Co-Optimization using Deep Reinforcement Learningcited by 29
  7. 2020Policy Transfer via Kinematic Domain Randomization and Adaptationcited by 16
  8. 2021Learning Inertial Odometry for Dynamic Legged Robot State Estimationcited by 16
  9. 2017Real-Time Motion Planning of Legged Robots: A Model Predictive Control Approachcited by 9
  10. 2023Actuator-Constrained Reinforcement Learning for High-Speed Quadrupedal Locomotioncited by 5
  11. 2021Knowledge Transfer across Imaging Modalities Via Simultaneous Learning of Adaptive Autoencoders for High-Fidelity Mobile Robot Visioncited by 1

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025Learning Massively Multitask World Models for Continuous Controlcited by 11
  2. 2026ZEST: Zero-shot Embodied Skill Transfer for Athletic Robot Controlcited by 4
  3. 2025Differentiable Weightless Controllers: Learning Logic Circuits for Continuous Controlcited by 1
  4. 2026LineRides: Line-Guided Reinforcement Learning for Bicycle Robot Stuntscited by 0
  5. 2026Mind the Phase: Effective Rank and Representation Health in Legged Locomotioncited by 0