mlsanityData to 7 Oct 2026

Compositional Learning-based Planning for Vision POMDPs

arXiv 2112.09456 ↗First posted 17 Dec 2021cs.AI cs.LG cs.ROcited by 8

Not on the map. Why?

Abstract (arXiv)

The Partially Observable Markov Decision Process (POMDP) is a powerful framework for capturing decision-making problems that involve state and transition uncertainty. However, most current POMDP planners cannot effectively handle high-dimensional image observations prevalent in real world applications, and often require lengthy online training that requires interaction with the environment. In this work, we propose Visual Tree Search (VTS), a compositional learning and planning procedure that combines generative models learned offline with online model-based POMDP planning. The deep generative observation models evaluate the likelihood of and predict future image observations in a Monte Carlo tree search planner. We show that VTS is robust to different types of image noises that were not present during training and can adapt to different reward structures without the need to re-train. This new approach significantly and stably outperforms several baseline state-of-the-art vision POMDP algorithms while using a fraction of the training time.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 33 of its 52 references that are in the map. First posted on arXiv 17 Dec 2021.

Builds on

33 of 52 references.

References · 33

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  2. 2014Conditional Generative Adversarial Netscited by 2,152
  3. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  4. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  5. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 404
  6. 2015Neural Module Networkscited by 375
  7. 2018Compositional Attention Networks for Machine Reasoningcited by 183
  8. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 138
  9. 2017Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learningcited by 125
  10. 2016Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfercited by 124
  11. 2018Deep Variational Reinforcement Learning for POMDPscited by 122
  12. 2016DESPOT: Online POMDP Planning with Regularizationcited by 105
21 more
  1. 2017Online algorithms for POMDPs with continuous state, action, and observation spacescited by 82
  2. 2018Modular Networks: Learning to Decompose Neural Computationcited by 62
  3. 2021Differentiable Particle Filtering via Entropy-Regularized Optimal Transportcited by 59
  4. 2018Differentiable Particle Filters: End-to-End Learning with Algorithmic Priorscited by 58
  5. 2017Towards Deeper Understanding of Variational Autoencoding Modelscited by 56
  6. 2017QMDP-Net: Deep Learning for Planning under Partial Observabilitycited by 51
  7. 2018Modular meta-learningcited by 51
  8. 2020Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modulescited by 38
  9. 2021Differentiable SLAM-net: Learning Particle SLAM for Visual Navigationcited by 34
  10. 2017Beyond Shared Hierarchies: Deep Multitask Learning through Soft Layer Orderingcited by 28
  11. 2019DualSMC: Tunneling Differentiable Filtering and Planning under Continuous POMDPscited by 21
  12. 2021Structured World Belief for Reinforcement Learning in POMDPcited by 21
  13. 2020Bayesian Optimized Monte Carlo Planningcited by 16
  14. 2017The Value of Inferring the Internal State of Traffic Participants for Autonomous Freeway Drivingcited by 15
  15. 2020An On-Line POMDP Solver for Continuous Observation Spacescited by 15
  16. 2021Multi-Task Learning with Sequence-Conditioned Transporter Networkscited by 8
  17. 2019Sparse tree search optimality guarantees in POMDPs with continuous observation spacescited by 7
  18. 2021Deep Visual Navigation under Partial Observabilitycited by 6
  19. 2017GP-ILQG: Data-driven Robust Optimal Control for Uncertain Nonlinear Dynamical Systemscited by 4
  20. 2018Integrating Algorithmic Planning and Deep Learning for Partially Observable Navigationcited by 3
  21. 2020Differentiable Mapping Networks: Learning Structured Map Representations for Sparse Visual Localizationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 1; 2025: 1; 2026: 4.

8 papers.

  1. 2022Optimality Guarantees for Particle Belief Approximation of POMDPscited by 20
  2. 2026VISION-SLS: Safe Perception-Based Control from Learned Visual Representations via System Level Synthesiscited by 3
  3. 2023Simplifying Complex Observation Models in Continuous POMDP Planning with Probabilistic Guarantees and Practicecited by 2
  4. 2025Sequential Monte Carlo for Policy Optimization in Continuous POMDPscited by 2
  5. 2024AgentForge: A Flexible Low-Code Platform for Reinforcement Learning Agent Designcited by 1
  6. 2026Perception-Based Beliefs for POMDPs with Visual Observationscited by 1
  7. 2026VOiLA: Vectorized Online Planning with Learned Diffusion Models for POMDP Agentscited by 0
  8. 2026Scalable Rao-Blackwellized Online Planning for High-Dimensional POMDPscited by 0