mlsanityData to 7 Oct 2026

SafeDreamer: Safe Reinforcement Learning with World Models

arXiv 2307.07176 ↗First posted 14 Jul 2023cs.LG cs.AI cs.CVcited by 50

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

The deployment of Reinforcement Learning (RL) in real-world applications is constrained by its failure to satisfy safety criteria. Existing Safe Reinforcement Learning (SafeRL) methods, which rely on cost functions to enforce safety, often fail to achieve zero-cost performance in complex scenarios, especially vision-only tasks. These limitations are primarily due to model inaccuracies and inadequate sample efficiency. The integration of the world model has proven effective in mitigating these shortcomings. In this work, we introduce SafeDreamer, a novel algorithm incorporating Lagrangian-based methods into world model planning processes within the superior Dreamer framework. Our method achieves nearly zero-cost performance on various tasks, spanning low-dimensional and vision-only input, within the Safety-Gymnasium benchmark, showcasing its efficacy in balancing performance and safety in RL tasks. Further details can be found in the code repository: urlhttps://github.com/PKU-Alignment/SafeDreamer.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 33 of its 61 references that are in the map. First posted on arXiv 14 Jul 2023.

Builds on

33 of 61 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Mastering Diverse Domains through World Models10 Jan
  2. 2022Constrained Policy Optimization via Bayesian World Models24 Jan
  3. 2020Mastering Atari with Discrete World Models5 Oct
  4. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec

Other references · 29

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  4. 2021Convex optimizationcited by 2,479
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  6. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  7. 2017Constrained Policy Optimizationcited by 928
  8. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  9. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  10. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  11. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  12. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 302
17 more
  1. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 281
  2. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 228
  3. 2020Projection-Based Constrained Policy Optimizationcited by 195
  4. 2018Learning-based Model Predictive Control for Safe Explorationcited by 116
  5. 2022Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learningcited by 113
  6. 2018A predictive safety filter for learning-based control of constrained nonlinear dynamical systemscited by 91
  7. 2023OmniSafe: An Infrastructure for Accelerating Safe Reinforcement Learning Researchcited by 76
  8. 2022Safe Reinforcement Learning by Imagining the Near Futurecited by 67
  9. 2022Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentationcited by 63
  10. 2022Constrained Update Projection Approach to Safe Policy Optimizationcited by 55
  11. 2020Learning Off-Policy with Online Planningcited by 46
  12. 2022Model-based Safe Deep Reinforcement Learning via a Constrained Proximal Policy Optimization Algorithmcited by 27
  13. 2021Joint Synthesis of Safety Certificate and Safe Control Policy using Constrained Reinforcement Learningcited by 25
  14. 2023AutoCost: Evolving Intrinsic Cost for Zero-violation Reinforcement Learningcited by 16
  15. 2021Conservative and Adaptive Penalty for Model-Based Safe Reinforcement Learningcited by 13
  16. 2022Safe Reinforcement Learning From Pixels Using a Stochastic Latent Representationcited by 11
  17. 2023ROSARL: Reward-Only Safe Reinforcement Learningcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 13; 2025: 14; 2026: 22.

50 papers.

Sort:

By area:

  1. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  2. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  3. 2024World Models for Autonomous Driving: An Initial Surveycited by 79
  4. 2024The Matrix: Infinite-Horizon World Generation with Real-Time Moving Controlcited by 79
  5. 2024LLM3:Large Language Model-based Task and Motion Planning with Motion Failure Reasoningcited by 63
  6. 2025A Survey of World Models for Autonomous Drivingcited by 60
  7. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Modelscited by 50
  8. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  9. 2024ActSafe: Active Exploration with Safety Constraints for Reinforcement Learningcited by 22
  10. 2025Latent Chain-of-Thought World Modeling for End-to-End Drivingcited by 14
  11. 2024Constrained Reinforcement Learning with Smoothed Log Barrier Functioncited by 12
  12. 2024Dynamic Model Predictive Shielding for Provably Safe Reinforcement Learningcited by 11
38 more
  1. 2024OASIS: Conditional Distribution Shaping for Offline Safe Reinforcement Learningcited by 10
  2. 2026Is the Future Compatible? Diagnosing Dynamic Consistency in World Action Modelscited by 10
  3. 2024Varying Shades of Wrong: Aligning LLMs with Wrong Answers Onlycited by 9
  4. 2025SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfercited by 8
  5. 2025Discrete GCBF Proximal Policy Optimization for Multi-agent Safe Optimal Controlcited by 7
  6. 2025PIGDreamer: Privileged Information Guided World Models for Safe Partially Observable Reinforcement Learningcited by 7
  7. 2024Robust Training of Federated Models with Extremely Label Deficiencycited by 6
  8. 2026Failure-Aware RL: Reliable Offline-to-Online Reinforcement Learning with Self-Recovery for Real-World Manipulationcited by 6
  9. 2026Safe Exploration via Policy Priorscited by 6
  10. 2026WMAttack: Automated Attack Search for Adversarial Evaluation of World-Model Agentscited by 6
  11. 2025Safety Representations for Safer Policy Learningcited by 4
  12. 2025Solving Multi-Agent Safe Optimal Control with Distributed Epigraph Form MARLcited by 4
  13. 2025Hierarchical Multi-Agent Reinforcement Learning with Control Barrier Functions for Safety-Critical Autonomous Systemscited by 4
  14. 2024Feasibility Consistent Representation Learning for Safe Reinforcement Learningcited by 3
  15. 2024FOSP: Fine-tuning Offline Safe Policy through World Modelscited by 3
  16. 2024WARPD: World model Assisted Reactive Policy Diffusioncited by 3
  17. 2025Edge General Intelligence Through World Models and Agentic AI: Fundamentals, Solutions, and Challengescited by 3
  18. 2025Object-Centric World Models for Causality-Aware Reinforcement Learningcited by 3
  19. 2025World Model Robustness via Surprise Recognitioncited by 3
  20. 2026Safety, Security, and Cognitive Risks in World Modelscited by 3
  21. 2026Morphology-Conditioned World Model for Cross-Embodiment Quadrupedal Locomotioncited by 3
  22. 2026Off-Policy Safe Reinforcement Learning with Constrained Optimistic Explorationcited by 2
  23. 2026PH-Dreamer: A Physics-Driven World Model via Port-Hamiltonian Generative Dynamicscited by 2
  24. 2026SHAPO: Sharpness-Aware Policy Optimization for Safe Explorationcited by 2
  25. 2026ARB4WM: An Adversarial Robustness Benchmark for World Models in Continuous Controlcited by 2
  26. 2026SafeDojo: Safe Reinforcement Learning for VLA via Interactive World Modelcited by 2
  27. 2025Offline vs. Online Learning in Model-based RL: Lessons for Data Collection Strategiescited by 1
  28. 2026LIDAR-AD: A Decoder-Free Latent-Interaction Dreamer with Action-Residual Chains for Autonomous Drivingcited by 1
  29. 2025Towards a Practical Understanding of Lagrangian Methods in Safe Reinforcement Learningcited by 0
  30. 2026A Co-Evolutionary Theory of Human-AI Coexistence: Mutualism, Governance, and Dynamics in Complex Societiescited by 0
  31. 2026Sampling-Based Safe Reinforcement Learningcited by 0
  32. 2026COP-Q: Safety-First Reinforcement Learning for Robot Control via Cholesky-Ordered Projectioncited by 0
  33. 2026Utility-Constrained Policy Optimizationcited by 0
  34. 2026Sensitivity Shaping for Latent Modelingcited by 0
  35. 2026Imagine to Ensure Safety in Hierarchical Reinforcement Learningcited by 0
  36. 2026Safe and Generalizable Hierarchical Multi-Agent RL via Constraint Manifold Controlcited by 0
  37. 2026Learning Safe Agent Behaviour from Human Preferences and Justifications via World Modelscited by 0
  38. 2026Q-learning Penalized Transformer for Safe Offline Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Safe Planning and Policy Optimization via World Model Learning
  2. 2026Nightmare Dreamer: Dreaming About Unsafe States And Planning Ahead
  3. 2025SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learning
  4. 2023SaFormer: A Conditional Sequence Modeling Approach to Offline Safe Reinforcement Learning
  5. 2023Datasets and Benchmarks for Offline Safe Reinforcement Learning
  6. 2023Safe Reinforcement Learning in a Simulated Robotic Arm
  7. 2024Safe Exploration Using Bayesian World Models and Log-Barrier Optimization
  8. 2026Beyond Hard Constraints: Budget-Conditioned Reachability For Safe Offline Reinforcement Learning