# ESCORT: Efficient Stein-variational and Sliced Consistency-Optimized Temporal Belief Representation for POMDPs

arXiv: https://arxiv.org/abs/2510.21107 · HTML: https://mlsanity.com/paper/2510.21107/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 41 references that are in the map.
- First posted on arXiv 24 Oct 2025.

## Builds on (most cited first, 21 of 21)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents](https://mlsanity.com/paper/1709.06009/) (2017)
- [Generalized Sliced Wasserstein Distances](https://mlsanity.com/paper/1902.00434/) (2019)
- [Stein Variational Gradient Descent as Gradient Flow](https://mlsanity.com/paper/1704.07520/) (2017)
- [Deep Variational Reinforcement Learning for POMDPs](https://mlsanity.com/paper/1806.02426/) (2018)
- [DESPOT: Online POMDP Planning with Regularization](https://mlsanity.com/paper/1609.03250/) (2016)
- [Partially Observable Markov Decision Processes in Robotics: A Survey](https://mlsanity.com/paper/2209.10342/) (2022)
- [Online algorithms for POMDPs with continuous state, action, and observation spaces](https://mlsanity.com/paper/1709.06196/) (2017)
- [On Improving Deep Reinforcement Learning for POMDPs](https://mlsanity.com/paper/1804.06309/) (2018)
- [Partially Observable Markov Decision Processes (POMDPs) and Robotics](https://mlsanity.com/paper/2107.07599/) (2021)
- [Message Passing Stein Variational Gradient Descent](https://mlsanity.com/paper/1711.04425/) (2017)
- [Projected Stein Variational Gradient Descent](https://mlsanity.com/paper/2002.03469/) (2020)
- [Discriminative Particle Filter Reinforcement Learning for Complex Partial Observations](https://mlsanity.com/paper/2002.09884/) (2020)
- [Tractable Inference for Complex Stochastic Processes](https://mlsanity.com/paper/1301.7362/) (2013)
- [Flow-based Recurrent Belief State Learning for POMDPs](https://mlsanity.com/paper/2205.11051/) (2022)
- [Fighting Sample Degeneracy and Impoverishment in Particle Filters: A Review of Intelligent Approaches](https://mlsanity.com/paper/1308.2443/) (2013)
- [Shrinking POMCP: A Framework for Real-Time UAV Search and Rescue](https://mlsanity.com/paper/2411.12967/) (2024)
- [Observation Adaptation via Annealed Importance Resampling for Partially Observable Markov Decision Processes](https://mlsanity.com/paper/2503.19302/) (2025)
- [Stein Variational Message Passing for Continuous Graphical Models](https://mlsanity.com/paper/1711.07168/) (2017)

## Built on by (most cited first, 1 of 1)

- [Stein Kernelized Molecular Dynamics for Active Learning of Interatomic Potentials](https://mlsanity.com/paper/2606.04100/) (2026)

## Abstract (arXiv)

In Partially Observable Markov Decision Processes (POMDPs), maintaining and updating belief distributions over possible underlying states provides a principled way to summarize action-observation history for effective decision-making under uncertainty. As environments grow more realistic, belief distributions develop complexity that standard mathematical models cannot accurately capture, creating a fundamental challenge in maintaining representational accuracy. Despite advances in deep learning and probabilistic modeling, existing POMDP belief approximation methods fail to accurately represent complex uncertainty structures such as high-dimensional, multi-modal belief distributions, resulting in estimation errors that lead to suboptimal agent behaviors. To address this challenge, we present ESCORT (Efficient Stein-variational and sliced Consistency-Optimized Representation for Temporal beliefs), a particle-based framework for capturing complex, multi-modal distributions in high-dimensional belief spaces. ESCORT extends SVGD with two key innovations: correlation-aware projections that model dependencies between state dimensions, and temporal consistency constraints that stabilize updates while preserving correlation structures. This approach retains SVGD's attractive-repulsive particle dynamics while enabling accurate modeling of intricate correlation patterns. Unlike particle filters prone to degeneracy or parametric methods with fixed representational capacity, ESCORT dynamically adapts to belief landscape complexity without resampling or restrictive distributional assumptions. We demonstrate ESCORT's effectiveness through extensive evaluations on both POMDP domains and synthetic multi-modal distributions of varying dimensionality, where it consistently outperforms state-of-the-art methods in terms of belief approximation accuracy and downstream decision quality.