# PIGDreamer: Privileged Information Guided World Models for Safe Partially Observable Reinforcement Learning

arXiv: https://arxiv.org/abs/2508.02159 · HTML: https://mlsanity.com/paper/2508.02159/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Latent dynamics models and model-based RL, one of 11 directions in World models.
- Builds on 23 of its 42 references that are in the map.
- First posted on arXiv 4 Aug 2025.

## Placed in

- World models / [Latent dynamics models and model-based RL](https://mlsanity.com/wm/latent-dynamics-models-and-model-based-rl/)
- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 23 of 23)

- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Deep Reinforcement Learning at the Edge of the Statistical Precipice](https://mlsanity.com/paper/2108.13264/) (2021)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [Safe Model-based Reinforcement Learning with Stability Guarantees](https://mlsanity.com/paper/1705.08551/) (2017)
- [Asymmetric Actor Critic for Image-Based Robot Learning](https://mlsanity.com/paper/1710.06542/) (2017)
- [Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model](https://mlsanity.com/paper/1907.00953/) (2019)
- [Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark](https://mlsanity.com/paper/2310.12567/) (2023)
- [Safe Reinforcement Learning by Imagining the Near Future](https://mlsanity.com/paper/2202.07789/) (2022)
- [Sample-Efficient Reinforcement Learning of Undercomplete POMDPs](https://mlsanity.com/paper/2006.12484/) (2020)
- [Suphx: Mastering Mahjong with Deep Reinforcement Learning](https://mlsanity.com/paper/2003.13590/) (2020)
- [SafeDreamer: Safe Reinforcement Learning with World Models](https://mlsanity.com/paper/2307.07176/) (2023)
- [Model-based Safe Deep Reinforcement Learning via a Constrained Proximal Policy Optimization Algorithm](https://mlsanity.com/paper/2210.07573/) (2022)
- [Unbiased Asymmetric Reinforcement Learning under Partial Observability](https://mlsanity.com/paper/2105.11674/) (2021)
- [TWIST: Teacher-Student World Model Distillation for Efficient Sim-to-Real Transfer](https://mlsanity.com/paper/2311.03622/) (2023)
- [Safe Continuous Control with Constrained Model-Based Policy Optimization](https://mlsanity.com/paper/2104.06922/) (2021)
- [GUARD: A Safe Reinforcement Learning Benchmark](https://mlsanity.com/paper/2305.13681/) (2023)
- [Provable Partially Observable Reinforcement Learning with Privileged Information](https://mlsanity.com/paper/2412.00985/) (2024)
- [Safe Reinforcement Learning From Pixels Using a Stochastic Latent Representation](https://mlsanity.com/paper/2210.01801/) (2022)
- [The Wasserstein Believer: Learning Belief Updates for Partially Observable Environments through Reliable Latent Space Models](https://mlsanity.com/paper/2303.03284/) (2023)
- [The Augmented Lagrangian Method Can Approximately Solve Convex Optimization with Least Constraint Violation](https://mlsanity.com/paper/2111.06194/) (2021)

## Built on by (most cited first, 7 of 7)

- [MetaMind: General and Cognitive World Models in Multi-Agent Systems by Meta-Theory of Mind](https://mlsanity.com/paper/2603.00808/) (2026)
- [Unifying Agent Interaction and World Information for Multi-agent Coordination](https://mlsanity.com/paper/2509.25550/) (2025)
- [Driving Beyond Privilege: Distilling Dense-Reward Knowledge into Sparse-Reward Policies](https://mlsanity.com/paper/2512.04279/) (2025)
- [Confounding Robust Continuous Control via Automatic Reward Shaping](https://mlsanity.com/paper/2602.10305/) (2026)
- [Discovering What You Can Control: Interventional Boundary Discovery for Reinforcement Learning](https://mlsanity.com/paper/2603.18257/) (2026)
- [JEPA-x: Cross-Predictive Physics Grounding for Forecastable Latent Dynamics](https://mlsanity.com/paper/2608.24044/) (2026)
- [Belief-State RWKV for Reinforcement Learning under Partial Observability](https://mlsanity.com/paper/2604.09671/) (2026)

## Abstract (arXiv)

Partial observability presents a significant challenge for Safe Reinforcement Learning (Safe RL), as it impedes the identification of potential risks and rewards. Leveraging specific types of privileged information during training to mitigate the effects of partial observability has yielded notable empirical successes. In this paper, we propose Asymmetric Constrained Partially Observable Markov Decision Processes (ACPOMDPs) to theoretically examine the advantages of incorporating privileged information in Safe RL. Building upon ACPOMDPs, we propose the Privileged Information Guided Dreamer (PIGDreamer), a model-based RL approach that leverages privileged information to enhance the agent's safety and performance through privileged representation alignment and an asymmetric actor-critic structure. Our empirical results demonstrate that PIGDreamer significantly outperforms existing Safe RL methods. Furthermore, compared to alternative privileged RL methods, our approach exhibits enhanced performance, robustness, and efficiency. Codes are available at: https://github.com/hggforget/PIGDreamer.