Data to 7 Oct 2026

Deep Variational Reinforcement Learning for POMDPs

arXiv 1806.02426 ↗First posted 6 Jun 2018cs.LG stat.MLcited by 122

Not on the map. Why?

Abstract (arXiv)

Many real-world sequential decision making problems are partially observable by nature, and the environment model is typically unknown. Consequently, there is great need for reinforcement learning methods that can tackle such problems given only a stream of incomplete and noisy observations. In this paper, we propose deep variational reinforcement learning (DVRL), which introduces an inductive bias that allows an agent to learn a generative model of the environment and perform inference in that model to effectively aggregate the available information. We develop an n-step approximation to the evidence lower bound (ELBO), allowing the model to be trained jointly with the policy. This ensures that the latent state representation is suitable for the control task. In experiments on Mountain Hike and flickering Atari we show that our method outperforms previous approaches relying on recurrent neural networks to encode the past.

Key facts. Cited by 122 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 25; 2022: 28; 2023: 23; 2024: 15; 2025: 18; 2026: 12.

122 papers.

Sort:

By area:

2022Flexible Diffusion Modeling of Long Videoscited by 294
2023Modern Bayesian Experimental Designcited by 121
2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
2022Visuo-Tactile Transformers for Manipulationcited by 43
2022Provable Reinforcement Learning with a Short-Term Memorycited by 40
2023Crossway Diffusion: Improving Diffusion-based Visuomotor Policy via Self-supervised Learningcited by 37
2022Deep Transformer Q-Networks for Partially Observable Reinforcement Learningcited by 35
2021Proper Value Equivalencecited by 33
2021Block Contextual MDPs for Continual Learningcited by 25
2022Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?cited by 24
2022The Free Energy Principle for Perception and Action: A Deep Learning Perspectivecited by 24
2019Learning Belief Representations for Imitation Learning in POMDPscited by 23

38 more

2022Bellman Meets Hawkes: Model-Based Reinforcement Learning via Temporal Point Processescited by 23
2021Bayesian Sequential Optimal Experimental Design for Nonlinear Models Using Policy Gradient Reinforcement Learningcited by 22
2021Structured World Belief for Reinforcement Learning in POMDPcited by 21
2021Sublinear Regret for Learning POMDPscited by 19
2021Recurrent Off-policy Baselines for Memory-based Continuous Controlcited by 19
2021Neural Recursive Belief States in Multi-Agent Reinforcement Learningcited by 18
2021Differentiable Particle Filtering without Modifying the Forward Passcited by 18
2021Sparsely Changing Latent States for Prediction and Planning in Partially Observable Domainscited by 18
2022Leveraging Fully Observable Policies for Learning under Partial Observabilitycited by 17
2024Equivariant Reinforcement Learning under Partial Observabilitycited by 17
2022Flow-based Recurrent Belief State Learning for POMDPscited by 16
2022A Temporal-Pattern Backdoor Attack to Deep Reinforcement Learningcited by 15
2022An Empirical Investigation of Representation Learning for Imitationcited by 15
2022Latent State Marginalization as a Low-cost Approach for Improving Explorationcited by 13
2023BetaZero: Belief-State Planning for Long-Horizon POMDPs using Learned Approximationscited by 13
2022Value Functions Factorization with Latent State Information Sharing in Decentralized Multi-Agent Policy Gradientscited by 12
2022ROI-Constrained Bidding via Curriculum-Guided Bayesian Reinforcement Learningcited by 12
2022A General Learning Framework for Open Ad Hoc Teamwork Using Graph-based Policy Learningcited by 12
2022Recurrent networks, hidden states and beliefs in partially observable environmentscited by 11
2023Informed POMDP: Leveraging Additional Information in Model-Based RLcited by 11
2022DGPO: Discovering Multiple Strategies with Diversity-Guided Policy Optimizationcited by 10
2022Reinforcement Learning in Non-Markovian Environmentscited by 10
2023Inverse Decision Modeling: Learning Interpretable Representations of Behaviorcited by 10
2023Provable Representation with Efficient Planning for Partial Observable Reinforcement Learningcited by 10
2021Training a Resilient Q-Network against Observational Interferencecited by 9
2022Reinforcement Learning in Presence of Discrete Markovian Context Evolutioncited by 9
2023The Wasserstein Believer: Learning Belief Updates for Partially Observable Environments through Reliable Latent Space Modelscited by 9
2025Discrete Codebook World Models for Continuous Controlcited by 9
2021Prior Preference Learning from Experts:Designing a Reward with Active Inferencecited by 8
2021Context-Specific Representation Abstraction for Deep Option Learningcited by 8
2021Compositional Learning-based Planning for Vision POMDPscited by 8
2023Bridging the Sim-to-Real Gap from the Information Bottleneck Perspectivecited by 8
2021Hierarchical and Partially Observable Goal-driven Policy Learning with Goals Relational Graphcited by 7
2021SIDE: State Inference for Partially Observable Cooperative Multi-Agent Reinforcement Learningcited by 7
2023Graph schemas as abstractions for transfer learning, inference, and planningcited by 7
2023Offline RL with Observation Histories: Analyzing and Improving Sample Complexitycited by 7
2024Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutionscited by 7
2023Seq2Seq Imitation Learning for Tactile Feedback-based Manipulationcited by 6

50 most cited of 122 All 122 (JSON)