mlsanityData to 7 Oct 2026

Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability

arXiv 2209.08025 ↗First posted 16 Sep 2022cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

A trustworthy reinforcement learning algorithm should be competent in solving challenging real-world problems, including {robustly} handling uncertainties, satisfying {safety} constraints to avoid catastrophic failures, and {generalizing} to unseen scenarios during deployments. This study aims to overview these main perspectives of trustworthy reinforcement learning considering its intrinsic vulnerabilities on robustness, safety, and generalizability. In particular, we give rigorous formulations, categorize corresponding methodologies, and discuss benchmarks for each perspective. Moreover, we provide an outlook section to spur promising future directions with a brief discussion on extrinsic vulnerabilities considering human feedback. We hope this survey could bring together separate threads of studies together in a unified framework and promote the trustworthiness of reinforcement learning.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 143 of its 223 references that are in the map. First posted on arXiv 16 Sep 2022.

Builds on

143 of 223 references.

References · 143

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  6. 2015Trust Region Policy Optimizationcited by 2,503
  7. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  8. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  9. 2017Hindsight Experience Replaycited by 1,096
  10. 2017Constrained Policy Optimizationcited by 928
  11. 2019Solving Rubik's Cube with a Robot Handcited by 699
  12. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
38 more
  1. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  2. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  3. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  4. 2017Robust Adversarial Reinforcement Learningcited by 439
  5. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  6. 2018Reward Constrained Policy Optimizationcited by 397
  7. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 388
  8. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 347
  9. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  10. 2017Certifying Some Distributional Robustness with Principled Adversarial Trainingcited by 333
  11. 2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 322
  12. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
  13. 2017Adversarial Attacks on Neural Network Policiescited by 314
  14. 2019Reinforcement Learning in Healthcare: A Surveycited by 308
  15. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 302
  16. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 294
  17. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 277
  18. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  19. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 260
  20. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 256
  21. 2018Towards Fast Computation of Certified Robustness for ReLU Networkscited by 254
  22. 2018Safe Exploration in Continuous Action Spacescited by 237
  23. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 235
  24. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 227
  25. 2015Contextual Markov Decision Processescited by 213
  26. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 209
  27. 2020Projection-Based Constrained Policy Optimizationcited by 195
  28. 2020Prioritized Level Replaycited by 190
  29. 2017Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulationcited by 182
  30. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
  31. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 167
  32. 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 163
  33. 2019Active Domain Randomizationcited by 161
  34. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 156
  35. 2021Embodied Intelligence via Learning and Evolutioncited by 154
  36. 2017Robust Deep Reinforcement Learning with Adversarial Attackscited by 152
  37. 2014Towards Deep Neural Network Architectures Robust to Adversarial Examplescited by 146
  38. 2020Exploration-Exploitation in Constrained MDPscited by 146

50 most cited of 143 All 143 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 16; 2024: 9; 2025: 3; 2026: 1.

32 papers.

Sort:

By area:

  1. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preferencecited by 252
  2. 2023Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmarkcited by 135
  3. 2023Constrained Decision Transformer for Offline Safe Reinforcement Learningcited by 65
  4. 2023Datasets and Benchmarks for Offline Safe Reinforcement Learningcited by 58
  5. 2022Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptationcited by 28
  6. 2023Safe Offline Reinforcement Learning with Real-Time Budget Constraintscited by 26
  7. 2023Soft Robust MDPs and Risk-Sensitive MDPs: Equivalence, Policy Gradient, and Sample Complexitycited by 16
  8. 2023Safety-aware Causal Representation for Trustworthy Offline Reinforcement Learning in Autonomous Drivingcited by 16
  9. 2023Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackerscited by 15
  10. 2023Learning Shared Safety Constraints from Multi-task Demonstrationscited by 15
  11. 2024Off-Policy Primal-Dual Safe Reinforcement Learningcited by 13
  12. 2024CaDRE: Controllable and Diverse Generation of Safety-Critical Driving Scenarios using Real-World Trajectoriescited by 11
20 more
  1. 2022Continual Vision-based Reinforcement Learning with Group Symmetriescited by 10
  2. 2024Dynamics as Prompts: In-Context Learning for Sim-to-Real System Identificationscited by 10
  3. 2023Optimal Transport Perturbations for Safe Reinforcement Learning with Robustness Guaranteescited by 9
  4. 2024Beyond the Edge: An Advanced Exploration of Reinforcement Learning for Mobile Edge Computing, its Applications, and Future Research Trajectoriescited by 9
  5. 2023Beyond Black-Box Advice: Learning-Augmented Algorithms for MDPs with Q-Value Predictionscited by 8
  6. 2023Gradient Shaping for Multi-Constraint Safe Reinforcement Learningcited by 7
  7. 2024Latent Safety-Constrained Policy Approach for Safe Offline Reinforcement Learningcited by 7
  8. 2022Group Distributionally Robust Reinforcement Learning with Hierarchical Latent Variablescited by 6
  9. 2023Communication-Robust Multi-Agent Learning by Adaptable Auxiliary Multi-Agent Adversary Generationcited by 6
  10. 2023Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challengescited by 5
  11. 2023Generalizable Resource Scaling of 5G Slices using Constrained Reinforcement Learningcited by 4
  12. 2023Robust Multi-agent Communication via Multi-view Message Certificationcited by 3
  13. 2023Stackelberg Driver Model for Continual Policy Improvement in Scenario-Based Closed-Loop Autonomous Drivingcited by 3
  14. 2024Feasibility Consistent Representation Learning for Safe Reinforcement Learningcited by 3
  15. 2024Safe Reinforcement Learning using Finite-Horizon Gradient-based Estimationcited by 2
  16. 2024Tolerance of Reinforcement Learning Controllers against Deviations in Cyber Physical Systemscited by 1
  17. 2025Offline Constrained Reinforcement Learning under Partial Data Coveragecited by 1
  18. 2025PAC-Bayesian Reinforcement Learning Trains Generalizable Policiescited by 1
  19. 2025Evaluating Reinforcement Learning Safety and Trustworthiness in Cyber-Physical Systemscited by 0
  20. 2026Why Does Agentic Safety Fail to Generalize Across Tasks?cited by 0