mlsanityData to 14 Oct 2026

Improving generalization of robot locomotion policies via Sharpness-Aware Reinforcement Learning

arXiv 2411.19732 ↗First posted 29 Nov 2024cs.RO cs.AI cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Reinforcement learning often requires extensive training data. Simulation-to-real transfer offers a promising approach to address this challenge in robotics. While differentiable simulators offer improved sample efficiency through exact gradients, they can be unstable in contact-rich environments and may lead to poor generalization. This paper introduces a novel approach integrating sharpness-aware optimization into gradient-based reinforcement learning algorithms. Our simulation results demonstrate that our method, tested on contact-rich environments, significantly enhances policy robustness to environmental variations and action perturbations while maintaining the sample efficiency of first-order methods. Specifically, our approach improves action noise tolerance compared to standard first-order methods and achieves generalization comparable to zeroth-order methods. This improvement stems from finding flatter minima in the loss landscape, associated with better generalization. Our work offers a promising solution to balance efficient learning and robust sim-to-real transfer in robotics, potentially bridging the gap between simulation and real-world performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 27 references that are in the map. First posted on arXiv 29 Nov 2024.

Builds on

14 of 27 references.

References · 14

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  3. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  4. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  5. 2018Learning Dexterous In-Hand Manipulationcited by 687
  6. 2019Learning agile and dynamic motor skills for legged robotscited by 504
  7. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  8. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 319
  9. 2021DiffPD: Differentiable Projective Dynamicscited by 62
  10. 2023Barkour: Benchmarking Animal-level Agility with Quadruped Robotscited by 41
  11. 2022Rethinking Optimization with Differentiable Simulation from a Global Perspectivecited by 23
  12. 2024Adaptive Horizon Actor-Critic for Policy Learning in Contact-Rich Differentiable Simulationcited by 20
2 more
  1. 2020ADD: Analytically Differentiable Dynamics for Multi-Body Systems with Frictional Contactcited by 17
  2. 2023Contact Models in Robotics: a Comparative Analysiscited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025An Real-Sim-Real (RSR) Loop Framework for Generalizable Robotic Policy Transfer
  2. 2026Physics-Informed Policy Optimization via Analytic Dynamics Regularization
  3. 2025Rapidly Adapting Policies to the Real World via Simulation-Guided Fine-Tuning
  4. 2025Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics
  5. 2026A Balanced Data Diet: Addressing the Exploration Bottleneck in Mega-Scale RL for Robot Control
  6. 2025Sim-to-Real Transfer in Deep Reinforcement Learning for Bipedal Locomotion
  7. 2026Does "Do Differentiable Simulators Give Better Policy Gradients?'' Give Better Policy Gradients?
  8. 2026SLowRL: Safe Low-Rank Adaptation Reinforcement Learning for Locomotion