mlsanityData to 7 Oct 2026

OmniSafe: An Infrastructure for Accelerating Safe Reinforcement Learning Research

arXiv 2305.09304 ↗First posted 16 May 2023cs.LG cs.AIcited by 76

Not on the map. Why?

Abstract (arXiv)

AI systems empowered by reinforcement learning (RL) algorithms harbor the immense potential to catalyze societal advancement, yet their deployment is often impeded by significant safety concerns. Particularly in safety-critical applications, researchers have raised concerns about unintended harms or unsafe behaviors of unaligned RL agents. The philosophy of safe reinforcement learning (SafeRL) is to align RL agents with harmless intentions and safe behavioral patterns. In SafeRL, agents learn to develop optimal policies by receiving feedback from the environment, while also fulfilling the requirement of minimizing the risk of unintended harm or unsafe behavior. However, due to the intricate nature of SafeRL algorithm implementation, combining methodologies across various domains presents a formidable challenge. This had led to an absence of a cohesive and efficacious learning framework within the contemporary SafeRL research milieu. In this work, we introduce a foundational framework designed to expedite SafeRL research endeavors. Our comprehensive framework encompasses an array of algorithms spanning different RL domains and places heavy emphasis on safety elements. Our efforts are to make the SafeRL-related research process more streamlined and efficient, therefore facilitating further research in AI safety. Our project is released at: https://github.com/PKU-Alignment/omnisafe.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). Builds on 27 of its 45 references that are in the map. First posted on arXiv 16 May 2023.

Builds on

27 of 45 references.

References · 27

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2015Trust Region Policy Optimizationcited by 2,503
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  7. 2017Constrained Policy Optimizationcited by 928
  8. 2018Reward Constrained Policy Optimizationcited by 397
  9. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
  10. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 302
  11. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 281
  12. 2020Critic Regularized Regressioncited by 269
15 more
  1. 2020Projection-Based Constrained Policy Optimizationcited by 195
  2. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
  3. 2021Tianshou: a Highly Modularized Deep Reinforcement Learning Librarycited by 139
  4. 2020CRPO: A New Approach for Safe Reinforcement Learning with Convergence Guaranteecited by 128
  5. 2019IPO: Interior-point Policy Optimization under Constraintscited by 119
  6. 2022COptiDICE: Offline Constrained Reinforcement Learning via Stationary Distribution Correction Estimationcited by 69
  7. 2022Safe Reinforcement Learning by Imagining the Near Futurecited by 67
  8. 2022Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentationcited by 63
  9. 2022Penalized Proximal Policy Optimization for Safe Reinforcement Learningcited by 60
  10. 2022Constrained Policy Optimization via Bayesian World Modelscited by 55
  11. 2022Constrained Update Projection Approach to Safe Policy Optimizationcited by 55
  12. 2020Learning Off-Policy with Online Planningcited by 46
  13. 2022Model-based Safe Deep Reinforcement Learning via a Constrained Proximal Policy Optimization Algorithmcited by 27
  14. 2021Safe Exploration by Solving Early Terminated MDPcited by 13
  15. 2021Conservative and Adaptive Penalty for Model-Based Safe Reinforcement Learningcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 21; 2025: 20; 2026: 29.

76 papers.

Sort:

By area:

  1. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  2. 2024Aligner: Efficient Alignment by Learning to Correctcited by 96
  3. 2025SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learningcited by 69
  4. 2024Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Modelcited by 62
  5. 2023Datasets and Benchmarks for Offline Safe Reinforcement Learningcited by 58
  6. 2023SafeDreamer: Safe Reinforcement Learning with World Modelscited by 50
  7. 2024A Survey of Constraint Formulations in Safe Reinforcement Learningcited by 42
  8. 2023Iterative Reachability Estimation for Safe Reinforcement Learningcited by 33
  9. 2025Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human Feedbackcited by 27
  10. 2024Balance Reward and Safety Optimization for Safe Reinforcement Learning: A Perspective of Gradient Manipulationcited by 21
  11. 2024DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RLcited by 20
  12. 2024MetaUrban: An Embodied AI Simulation Platform for Urban Micromobilitycited by 19
38 more
  1. 2025Mitigating Reward Over-Optimization in RLHF via Behavior-Supported Regularizationcited by 14
  2. 2024Off-Policy Primal-Dual Safe Reinforcement Learningcited by 13
  3. 2024Constrained Reinforcement Learning with Smoothed Log Barrier Functioncited by 12
  4. 2025Probabilistic Shielding for Safe Reinforcement Learningcited by 12
  5. 2025A Survey of Safe Reinforcement Learning and Constrained MDPs: A Technical Survey on Single-Agent and Multi-Agent Safetycited by 12
  6. 2024Temporal Logic Specification-Conditioned Decision Transformer for Offline Safe Reinforcement Learningcited by 10
  7. 2024Safe and Balanced: A Framework for Constrained Multi-Objective Reinforcement Learningcited by 9
  8. 2024Offline Safe Reinforcement Learning Using Trajectory Classificationcited by 9
  9. 2024Enhancing Efficiency of Safe Reinforcement Learning via Sample Manipulationcited by 8
  10. 2024SoNIC: Safe Social Navigation with Adaptive Conformal Inference and Constrained Reinforcement Learningcited by 8
  11. 2024Improving Policy Optimization via varepsilon-Retraincited by 5
  12. 2025Towards Generalizable Safety in Crowd Navigation via Conformal Uncertainty Handlingcited by 5
  13. 2025Safety Representations for Safer Policy Learningcited by 4
  14. 2025HASARD: A Benchmark for Vision-Based Safe Reinforcement Learning in Embodied Agentscited by 4
  15. 2023Learning Adaptive Safety for Multi-Agent Systemscited by 3
  16. 2024What Hides behind Unfairness? Exploring Dynamics Fairness in Reinforcement Learningcited by 3
  17. 2024Flipping-based Policy for Chance-Constrained Markov Decision Processescited by 3
  18. 2025Designing Control Barrier Function via Probabilistic Enumeration for Safe Reinforcement Learning Navigationcited by 3
  19. 2026Extreme Value Policy Optimization for Safe Reinforcement Learningcited by 3
  20. 2024e-COP : Episodic Constrained Optimization of Policiescited by 2
  21. 2024Handling Long-Term Safety and Uncertainty in Safe Reinforcement Learningcited by 2
  22. 2024Safe Reinforcement Learning using Finite-Horizon Gradient-based Estimationcited by 2
  23. 2025Risk-Aware Reinforcement Learning for Autonomous Driving: Improving Safety When Driving through Intersectioncited by 2
  24. 2025Adaptive Shielding for Safe Reinforcement Learning under Hidden-Parameter Dynamics Shiftscited by 2
  25. 2025LUCID: Learning-Enabled Uncertainty-Aware Certification of Stochastic Dynamical Systemscited by 2
  26. 2026MoralityGym: A Benchmark for Evaluating Hierarchical Moral Alignment in Sequential Decision-Making Agentscited by 2
  27. 2026Off-Policy Safe Reinforcement Learning with Constrained Optimistic Explorationcited by 2
  28. 2026SHAPO: Sharpness-Aware Policy Optimization for Safe Explorationcited by 2
  29. 2024Concurrent Learning of Policy and Unknown Safety Constraints in Reinforcement Learningcited by 1
  30. 2025Depth-Constrained ASV Navigation with Deep RL and Limited Sensingcited by 1
  31. 2025Offline Safe Policy Optimization From Heterogeneous Feedbackcited by 1
  32. 2026Safe Heterogeneous Multi-Agent RL with Communication Regularization for Coordinated Target Acquisitioncited by 1
  33. 2026Safety Generalization Under Distribution Shift in Safe Reinforcement Learning: A Diabetes Testbedcited by 1
  34. 2026Safe Langevin Soft Actor Criticcited by 1
  35. 2026Safe Reinforcement Learning via Recovery-based Shielding with Gaussian Process Dynamics Modelscited by 1
  36. 2026SteinGate: Tail-Sensitive Safe Reinforcement Learning via Stein Discrepancycited by 1
  37. 2026Vision--Language Signals in Constrained RL: Safety Gains Without Anticipationcited by 1
  38. 2023Learning Safety Constraints From Demonstration Using One-Class Decision Treescited by 0

50 most cited of 76 All 76 (JSON)