mlsanityData to 14 Oct 2026

Variational Recurrent Models for Solving Partially Observable Control Tasks

arXiv 1912.10703 ↗First posted 23 Dec 2019cs.LG cs.NE cs.SYcited by 47

Not on the map. Why?

Abstract (arXiv)

In partially observable (PO) environments, deep reinforcement learning (RL) agents often suffer from unsatisfactory performance, since two problems need to be tackled together: how to extract information from the raw observations to solve the task, and how to improve the policy. In this study, we propose an RL algorithm for solving PO tasks. Our method comprises two parts: a variational recurrent model (VRM) for modeling the environment, and an RL controller that has access to both the environment and the VRM. The proposed algorithm was tested in two types of PO robotic control tasks, those in which either coordinates or velocities were not observable and those that require long-term memorization. Our experiments show that the proposed algorithm achieved better data efficiency and/or learned more optimal policy than other alternative approaches in tasks in which unobserved states cannot be inferred from raw observations in a simple manner.

Key facts. Cited by 47 papers in the map (data through 14 Oct 2026). First posted on arXiv 23 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 9; 2022: 7; 2023: 9; 2024: 7; 2025: 8; 2026: 6.

47 papers.

Sort:

By area:

  1. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 122
  2. 2020Offline Reinforcement Learning from Images with Latent Space Modelscited by 95
  3. 2023When Do Transformers Shine in RL? Decoupling Memory from Credit Assignmentcited by 63
  4. 2024Bridging State and History Representations: Understanding Self-Predictive RLcited by 54
  5. 2022VRL3: A Data-Driven Framework for Visual Deep Reinforcement Learningcited by 52
  6. 2022Learning in Observable POMDPs, without Computationally Intractable Oraclescited by 30
  7. 2022DreamingV2: Reinforcement Learning with Discrete World Models without Reconstructioncited by 29
  8. 2021Block Contextual MDPs for Continual Learningcited by 25
  9. 2021Recurrent Off-policy Baselines for Memory-based Continuous Controlcited by 21
  10. 2022Leveraging Fully Observable Policies for Learning under Partial Observabilitycited by 18
  11. 2024Equivariant Reinforcement Learning under Partial Observabilitycited by 18
  12. 2022Flow-based Recurrent Belief State Learning for POMDPscited by 16
35 more
  1. 2022An Adaptive Deep RL Method for Non-Stationary Environments with Piecewise Stable Contextcited by 15
  2. 2022Latent State Marginalization as a Low-cost Approach for Improving Explorationcited by 13
  3. 2023Informed POMDP: Leveraging Additional Information in Model-Based RLcited by 11
  4. 2023Inverse Decision Modeling: Learning Interpretable Representations of Behaviorcited by 10
  5. 2024Efficient Recurrent Off-Policy RL Requires a Context-Encoder-Specific Learning Ratecited by 10
  6. 2024Real-Time Recurrent Learning using Trace Units in Reinforcement Learningcited by 10
  7. 2021Hierarchical and Partially Observable Goal-driven Policy Learning with Goals Relational Graphcited by 7
  8. 2021SIDE: State Inference for Partially Observable Cooperative Multi-Agent Reinforcement Learningcited by 7
  9. 2023ODE-based Recurrent Model-free Reinforcement Learning for POMDPscited by 7
  10. 2025Habitizing Diffusion Planning for Efficient and Effective Decision Makingcited by 7
  11. 2025Benchmarking Partial Observability in Reinforcement Learning with a Suite of Memory-Improvable Domainscited by 7
  12. 2023Seq2Seq Imitation Learning for Tactile Feedback-based Manipulationcited by 6
  13. 2025A Theoretical Justification for Asymmetric Actor-Critic Algorithmscited by 6
  14. 2024On Diffusion Models for Multi-Agent Partial Observability: Shared Attractors, Error Bounds, and Composite Flowcited by 5
  15. 2021A Whole Brain Probabilistic Generative Model: Toward Realizing Cognitive Architectures for Developmental Robotscited by 4
  16. 2023Habits and goals in synergy: a variational Bayesian framework for behaviorcited by 4
  17. 2023PID-Inspired Inductive Biases for Deep Reinforcement Learning in Partially Observable Control Taskscited by 4
  18. 2024Beyond Local Views: Global State Inference with Diffusion Models for Cooperative Multi-Agent Reinforcement Learningcited by 4
  19. 2025Sequential Monte Carlo for Policy Optimization in Continuous POMDPscited by 4
  20. 2024GRSN: Gated Recurrent Spiking Neurons for POMDPs and MARLcited by 3
  21. 2021Estimating Disentangled Belief about Hidden State and Hidden Task for Meta-RLcited by 2
  22. 2021Goal-Directed Planning by Reinforcement Learning and Active Inferencecited by 2
  23. 2025Informed Asymmetric Actor-Critic: Leveraging Privileged Signals Beyond Full-State Accesscited by 2
  24. 2023Filter-Aware Model-Predictive Controlcited by 1
  25. 2025Synthetic POMDPs to Challenge Memory-Augmented RL: Memory Demand Structure Modelingcited by 1
  26. 2026Direct Advantage Estimation for Scalable and Sample-efficient Deep Reinforcement Learningcited by 1
  27. 2021Blockwise Sequential Model Learning for Partially Observable Reinforcement Learningcited by 0
  28. 2023Fast exploration and learning of latent graphs with aliased observationscited by 0
  29. 2025A New Perspective on Transformers in Online Reinforcement Learning for Continuous Controlcited by 0
  30. 2025Predictive Coding Enhances Meta-RL To Achieve Interpretable Bayes-Optimal Belief Representation Under Partial Observabilitycited by 0
  31. 2026State-Action Inpainting Diffuser for Continuous Control with Delaycited by 0
  32. 2026Hidden States as Value Gradients: The Pontryagin Structure of Recurrent Policiescited by 0
  33. 2026FlowMo-WM: A World Model with Object Momentum and Hidden Ambient Driftcited by 0
  34. 2026Reinformed Dreamer: An Asymmetric World Model Efficiently Trained through Latent Guidancecited by 0
  35. 2026Reinforcement Learning from Real-Robot Interaction without High-Quality State Estimatescited by 0