mlsanityData to 14 Oct 2026

SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfer

arXiv 2509.18648 ↗First posted 23 Sep 2025cs.RO cs.AIcited by 8

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Deploying reinforcement learning (RL) safely in the real world is challenging, as policies trained in simulators must face the inevitable sim-to-real gap. Robust safe RL techniques are provably safe, however difficult to scale, while domain randomization is more practical yet prone to unsafe behaviors. We address this gap by proposing SPiDR, short for Sim-to-real via Pessimistic Domain Randomization -- a scalable algorithm with provable guarantees for safe sim-to-real transfer. SPiDR uses domain randomization to incorporate the uncertainty about the sim-to-real gap into the safety constraints, making it versatile and highly compatible with existing training pipelines. Through extensive experiments on sim-to-sim benchmarks and two distinct real-world robotic platforms, we demonstrate that SPiDR effectively ensures safety despite the sim-to-real gap while maintaining strong performance.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 70 of its 102 references that are in the map. First posted on arXiv 23 Sep 2025.

Builds on

70 of 102 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024ActSafe: Active Exploration with Safety Constraints for Reinforcement Learning12 Oct
  2. 2024Balance Reward and Safety Optimization for Safe Reinforcement Learning: A Perspective of Gradient Manipulation2 May
  3. 2024Constrained Reinforcement Learning Under Model Mismatch2 May

Other references · 67

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2016Concrete Problems in AI Safetycited by 1,642
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  5. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  6. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  7. 2020MOPO: Model-based Offline Policy Optimizationcited by 636
  8. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  9. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  10. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  11. 2017Robust Adversarial Reinforcement Learningcited by 451
  12. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
38 more
  1. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  2. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  3. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  4. 2018Maximum a Posteriori Policy Optimisationcited by 328
  5. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  6. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 233
  7. 2020Projection-Based Constrained Policy Optimizationcited by 203
  8. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 181
  9. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 175
  10. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 174
  11. 2017Robust Deep Reinforcement Learning with Adversarial Attackscited by 154
  12. 2020CRPO: A New Approach for Safe Reinforcement Learning with Convergence Guaranteecited by 134
  13. 2020Learning to be Safe: Deep RL with a Safety Criticcited by 116
  14. 2020Conservative Safety Critics for Explorationcited by 113
  15. 2024Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous controlcited by 108
  16. 2020Fast active learning for pure exploration in reinforcement learningcited by 92
  17. 2021Understanding Domain Randomization for Sim-to-real Transfercited by 80
  18. 2020An empirical investigation of the challenges of real-world reinforcement learningcited by 77
  19. 2021RL for Latent MDPs: Regret Guarantees and a Lower Boundcited by 70
  20. 2022Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentationcited by 69
  21. 2022Safe Reinforcement Learning by Imagining the Near Futurecited by 67
  22. 2025Robust Autonomy Emerges from Self-Playcited by 67
  23. 2022Constrained Update Projection Approach to Safe Policy Optimizationcited by 58
  24. 2022Constrained Policy Optimization via Bayesian World Modelscited by 57
  25. 2023SafeDreamer: Safe Reinforcement Learning with World Modelscited by 54
  26. 2020Partial Policy Iteration for L1-Robust Markov Decision Processescited by 50
  27. 2025Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoidscited by 48
  28. 2023Not Only Rewards But Also Constraints: Applications on Legged Robot Locomotioncited by 47
  29. 2024GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPScited by 45
  30. 2018Variance-Aware Regret Bounds for Undiscounted Reinforcement Learning in MDPscited by 43
  31. 2019Distributionally Robust Reinforcement Learningcited by 42
  32. 2020Distributional Robustness and Regularization in Reinforcement Learningcited by 40
  33. 2020Planning in Markov Decision Processes with Gap-Dependent Sample Complexitycited by 34
  34. 2019AMZ Driverless: The Full Autonomous Racing Systemcited by 29
  35. 2022Sim-to-Lab-to-Real: Safe Reinforcement Learning with Shielding and Generalization Guaranteescited by 27
  36. 2020Deep Reinforcement Learning for Closed-Loop Blood Glucose Controlcited by 26
  37. 2023Seeing is not Believing: Robust Reinforcement Learning against Spurious Correlationcited by 25
  38. 2020Robust Constrained-MDPs: Soft-Constrained Robust Policy Optimization under Model Uncertaintycited by 23

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026Safe Exploration via Policy Priorscited by 6
  2. 2026Constrained Meta Reinforcement Learning with Provable Test-Time Safetycited by 1
  3. 2026Constraint-Aware Diffusion Priors for High-Fidelity and Versatile Quadruped Locomotioncited by 1
  4. 2026Safety Guarantees in Zero-Shot Reinforcement Learning for Cascade Dynamical Systemscited by 0
  5. 2026Reinforcement Learning Enabled Adaptive Multi-Task Control for Bipedal Soccer Robotscited by 0
  6. 2026Sampling-Based Safe Reinforcement Learningcited by 0
  7. 2026Transferable Reinforcement Learning via Probabilistic Latent Embeddings and Dynamic Policy Adaptation for Sim-to-Real Deploymentcited by 0
  8. 2026Provably Safe Sim-to-Real Transfercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Safe Continual Domain Adaptation after Sim2Real Transfer of Reinforcement Learning Policies in Robotics
  2. 2025Safe Domain Randomization via Uncertainty-Aware Out-of-Distribution Detection and Policy Adaptation
  3. 2024Safe and Robust Reinforcement Learning: Principles and Practice
  4. 2025Can Context Bridge the Reality Gap? Sim-to-Real Transfer of Context-Aware Policies
  5. 2025SafeOR-Gym: A Benchmark Suite for Safe Reinforcement Learning Algorithms on Practical Operations Research Problems
  6. 2026Demonstration-Guided Observation Attacks on Black-Box Safe Reinforcement Learning Controllers for Robotic Systems
  7. 2025An Real-Sim-Real (RSR) Loop Framework for Generalizable Robotic Policy Transfer
  8. 2024Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems