mlsanityData to 7 Oct 2026

Optimal Transport Perturbations for Safe Reinforcement Learning with Robustness Guarantees

arXiv 2301.13375 ↗First posted 31 Jan 2023cs.LG cs.AI stat.MLcited by 9

Not on the map. Why?

Abstract (arXiv)

Robustness and safety are critical for the trustworthy deployment of deep reinforcement learning. Real-world decision making applications require algorithms that can guarantee robust performance and safety in the presence of general environment disturbances, while making limited assumptions on the data collection process during training. In order to accomplish this goal, we introduce a safe reinforcement learning framework that incorporates robustness through the use of an optimal transport cost uncertainty set. We provide an efficient implementation based on applying Optimal Transport Perturbations to construct worst-case virtual state transitions, which does not impact data collection during training and does not require detailed simulator access. In experiments on continuous control tasks with safety constraints, our approach demonstrates robust performance while significantly improving safety at deployment time compared to standard safe reinforcement learning.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 39 of its 54 references that are in the map. First posted on arXiv 31 Jan 2023.

Builds on

39 of 54 references.

References · 39

  1. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  2. 2017Constrained Policy Optimizationcited by 928
  3. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  4. 2017Robust Adversarial Reinforcement Learningcited by 439
  5. 2018Reward Constrained Policy Optimizationcited by 397
  6. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  7. 2018Maximum a Posteriori Policy Optimisationcited by 318
  8. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 302
  9. 2019Challenges of Real-World Reinforcement Learningcited by 282
  10. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 256
  11. 2018Safe Exploration in Continuous Action Spacescited by 237
  12. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 235
27 more
  1. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 167
  2. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 156
  3. 2020Acme: A Research Framework for Distributed Reinforcement Learningcited by 146
  4. 2019Constrained Reinforcement Learning Has Zero Duality Gapcited by 144
  5. 2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 136
  6. 2020CRPO: A New Approach for Safe Reinforcement Learning with Convergence Guaranteecited by 128
  7. 2020Learning to be Safe: Deep RL with a Safety Criticcited by 109
  8. 2020Conservative Safety Critics for Explorationcited by 107
  9. 2019Robust Reinforcement Learning for Continuous Control with Model Misspecificationcited by 76
  10. 2020An empirical investigation of the challenges of real-world reinforcement learningcited by 75
  11. 2022Constrained Variational Policy Optimization for Safe Reinforcement Learningcited by 69
  12. 2020Robust Reinforcement Learning using Adversarial Populationscited by 64
  13. 2020A Distributional View on Multi-Objective Policy Optimizationcited by 51
  14. 2019Wasserstein Robust Reinforcement Learningcited by 50
  15. 2021Safe Reinforcement Learning Using Advantage-Based Interventioncited by 45
  16. 2021Distributionally Robust Learningcited by 44
  17. 2022On the Robustness of Safe Reinforcement Learning under Observational Perturbationscited by 38
  18. 2018Soft-Robust Actor-Critic Policy-Gradientcited by 37
  19. 2019Adversarial Risk via Optimal Transport and Optimal Couplingscited by 36
  20. 2015Distributionally Robust Counterpart in Markov Decision Processescited by 32
  21. 2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 32
  22. 2021Model-based Constrained Reinforcement Learning using Generalized Control Barrier Functioncited by 30
  23. 2020Robust Reinforcement Learning with Wasserstein Constraintcited by 18
  24. 2021Learning Robust Policy against Disturbance in Transition Dynamics via State-Conservative Policy Optimizationcited by 18
  25. 2020Robust Constrained Reinforcement Learning for Continuous Control with Model Misspecificationcited by 16
  26. 2023Risk-Averse Model Uncertainty for Distributionally Robust Safe Reinforcement Learningcited by 12
  27. 2021Lyapunov Robust Constrained-MDPs: Soft-Constrained Robustly Stable Policy Optimization under Model Uncertaintycited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 4; 2026: 2.

9 papers.

  1. 2025A Survey of Reinforcement Learning for Optimization in Automationcited by 4
  2. 2024Optimal Transport-Assisted Risk-Sensitive Q-Learningcited by 3
  3. 2024GRAM: Generalization in Deep RL with a Robust Adaptation Modulecited by 2
  4. 2025DR-PETS: Learning-Based Control With Planning in Adversarial Environmentscited by 1
  5. 2025Wasserstein Barycenter Soft Actor-Criticcited by 1
  6. 2024A Model-Based Approach for Improving Reinforcement Learning Efficiency Leveraging Expert Observationscited by 0
  7. 2025Optimal Transport-Guided Safety in Temporal Difference Reinforcement Learningcited by 0
  8. 2026C-STEP: Continuous Space-Time Empowerment for Physics-informed Safe Reinforcement Learning of Mobile Agentscited by 0
  9. 2026Post-Hoc Robustness for Model-Based Reinforcement Learningcited by 0