Informed POMDP: Leveraging Additional Information in Model-Based RL
On the map: World models → Latent dynamics models and model-based RL
Abstract (arXiv)
In this work, we generalize the problem of learning through interaction in a POMDP by accounting for eventual additional information available at training time. First, we introduce the informed POMDP, a new learning paradigm offering a clear distinction between the information at training and the observation at execution. Next, we propose an objective that leverages this information for learning a sufficient statistic of the history for the optimal control. We then adapt this informed objective to learn a world model able to sample latent trajectories. Finally, we empirically show a learning speed improvement in several environments using this informed world model in the Dreamer algorithm. These results and the simplicity of the proposed adaptation advocate for a systematic consideration of eventual additional information when learning in a POMDP using model-based RL.
Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 43 of its 54 references that are in the map. First posted on arXiv 20 Jun 2023.
Builds on
43 of 54 references.
Closest ancestors · 5
Same direction, newest first
- 2023Mastering Diverse Domains through World Models10 Jan
- 2020Mastering Atari with Discrete World Models5 Oct
- 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec
- 2018Recurrent World Models Facilitate Policy Evolution4 Sep
- 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models30 May
Other references · 38
- 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
- 2018DeepMind Control Suitecited by 956
- 2017Counterfactual Multi-Agent Policy Gradientscited by 751
- 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
- 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
- 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 379
- 2015A Recurrent Latent Variable Model for Sequential Datacited by 362
- 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 270
- 2017Asymmetric Actor Critic for Image-Based Robot Learningcited by 238
- 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 231
- 2020Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 197
26 more
- 2020FACMAC: Factored Multi-Agent Centralised Policy Gradientscited by 178
- 2019Unsupervised State Representation Learning in Ataricited by 144
- 2018Deep Variational Reinforcement Learning for POMDPscited by 125
- 2020Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learningcited by 102
- 2015Memory-based control with recurrent neural networkscited by 78
- 2020Approximate information state for approximate planning and reinforcement learning in partially observed systemscited by 70
- 2019Shaping Belief States with Generative Environment Models for RLcited by 66
- 2023POPGym: Benchmarking Partially Observable Reinforcement Learningcited by 62
- 2018On Improving Deep Reinforcement Learning for POMDPscited by 50
- 2018Learning and Querying Fast Generative Models for Reinforcement Learningcited by 48
- 2019Variational Recurrent Models for Solving Partially Observable Control Taskscited by 47
- 2018Neural Predictive Belief Representationscited by 40
- 2017Data-driven Planning via Imitation Learningcited by 36
- 2020R-MADDPG for Partially Observable Environments and Limited Communicationcited by 36
- 2020Discriminative Particle Filter Reinforcement Learning for Complex Partial Observationscited by 28
- 2021Unbiased Asymmetric Reinforcement Learning under Partial Observabilitycited by 25
- 2022Scalable Multi-Agent Model-Based Reinforcement Learningcited by 25
- 2020Robust Asymmetric Learning in POMDPscited by 23
- 2022Leveraging Fully Observable Policies for Learning under Partial Observabilitycited by 18
- 2022Rethinking Individual Global Max in Cooperative Multi-Agent Reinforcement Learningcited by 13
- 2022Recurrent networks, hidden states and beliefs in partially observable environmentscited by 12
- 2020Belief-Grounded Networks for Accelerated Robot Learning under Partial Observabilitycited by 10
- 2022A Deeper Understanding of State-Based Critics in Multi-Agent Reinforcement Learningcited by 10
- 2023The Wasserstein Believer: Learning Belief Updates for Partially Observable Environments through Reliable Latent Space Modelscited by 9
- 2020QVMix and QVMix-Max: Extending the Deep Quality-Value Family of Algorithms to Cooperative Multi-Agent Reinforcement Learningcited by 5
- 2022Mask Atari for Deep Reinforcement Learning as POMDP Benchmarkscited by 5
Built on by
Citing papers per year: 2024: 3; 2025: 6; 2026: 2.
11 papers.
- 2024Privileged Sensing Scaffolds Reinforcement Learningcited by 19
- 2025SkyDreamer: Interpretable End-to-End Vision-Based Drone Racing with Model-Based Reinforcement Learningcited by 8
- 2025A Theoretical Justification for Asymmetric Actor-Critic Algorithmscited by 6
- 2024Make the Pertinent Salient: Task-Relevant Reconstruction for Visual Control with Distractionscited by 5
- 2024An Empirical Study on the Power of Future Prediction in Partially Observable Environmentscited by 4
- 2025Informed Asymmetric Actor-Critic: Leveraging Privileged Signals Beyond Full-State Accesscited by 2
- 2025Real-World Reinforcement Learning of Active Perception Behaviorscited by 1
- 2025A Multi-Agent Reinforcement Learning Approach for Cooperative Air-Ground-Human Crowdsensing in Emergency Rescuecited by 0
- 2025Controllable Flow Matching for Online Reinforcement Learningcited by 0
- 2026Reinformed Dreamer: An Asymmetric World Model Efficiently Trained through Latent Guidancecited by 0
- 2026From Static Policies to Adaptive Priors in Offline Reinforcement Learningcited by 0
- 2026From Static Policies to Adaptive Priors in Offline Reinforcement Learning26 Sep 2026
- 2026Reinformed Dreamer: An Asymmetric World Model Efficiently Trained through Latent Guidance28 Jul 2026
- 2025Real-World Reinforcement Learning of Active Perception Behaviors1 Dec 2025
- 2025Controllable Flow Matching for Online Reinforcement Learning10 Nov 2025
- 2025SkyDreamer: Interpretable End-to-End Vision-Based Drone Racing with Model-Based Reinforcement Learning16 Oct 2025
- 2025Informed Asymmetric Actor-Critic: Leveraging Privileged Signals Beyond Full-State Access30 Sep 2025
- 2025A Multi-Agent Reinforcement Learning Approach for Cooperative Air-Ground-Human Crowdsensing in Emergency Rescue11 May 2025
- 2025A Theoretical Justification for Asymmetric Actor-Critic Algorithms31 Jan 2025
- 2024Make the Pertinent Salient: Task-Relevant Reconstruction for Visual Control with Distractions13 Oct 2024
- 2024Privileged Sensing Scaffolds Reinforcement Learning23 May 2024
- 2024An Empirical Study on the Power of Future Prediction in Partially Observable Environments11 Feb 2024
Similar, not cited
Close in wording, no citation either way
- 2026Learning POMDP World Models from Observations with Language-Model Priors
- 2025Synthetic POMDPs to Challenge Memory-Augmented RL: Memory Demand Structure Modeling
- 2024Adaptive World Models: Learning Behaviors by Latent Imagination Under Non-Stationarity
- 2026Inference Time Policy Optimization for Offline RL with Differentiable World Models
- 2024OffRIPP: Offline RL-based Informative Path Planning
- 2024PWM: Policy Learning with Multi-Task World Models
- 2020Ready Policy One: World Building Through Active Learning
- 2025Model-Based Reinforcement Learning under Random Observation Delays