# Efficient Sampling in POMDPs with Lipschitz Bandits for Motion Planning in Continuous Spaces

arXiv: https://arxiv.org/abs/2106.04206 · HTML: https://mlsanity.com/paper/2106.04206/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 36 references that are in the map.
- First posted on arXiv 8 Jun 2021.

## Builds on (most cited first, 8 of 8)

- [Introduction to Multi-Armed Bandits](https://mlsanity.com/paper/1904.07272/) (2019)
- [A General and Adaptive Robust Loss Function](https://mlsanity.com/paper/1701.03077/) (2017)
- [DESPOT: Online POMDP Planning with Regularization](https://mlsanity.com/paper/1609.03250/) (2016)
- [Lipschitz Continuity in Model-based Reinforcement Learning](https://mlsanity.com/paper/1804.07193/) (2018)
- [Online algorithms for POMDPs with continuous state, action, and observation spaces](https://mlsanity.com/paper/1709.06196/) (2017)
- [The Value of Inferring the Internal State of Traffic Participants for Autonomous Freeway Driving](https://mlsanity.com/paper/1702.00858/) (2017)
- [Belief State Planning for Autonomously Navigating Urban Intersections](https://mlsanity.com/paper/1704.04322/) (2017)
- [Adaptive-treed bandits](https://mlsanity.com/paper/1302.2489/) (2013)

## Built on by (most cited first, 2 of 2)

- [Quantum Lipschitz Bandits](https://mlsanity.com/paper/2504.02251/) (2025)
- [Generative AI for Autonomous Driving: A Review](https://mlsanity.com/paper/2505.15863/) (2025)

## Abstract (arXiv)

Decision making under uncertainty can be framed as a partially observable Markov decision process (POMDP). Finding exact solutions of POMDPs is generally computationally intractable, but the solution can be approximated by sampling-based approaches. These sampling-based POMDP solvers rely on multi-armed bandit (MAB) heuristics, which assume the outcomes of different actions to be uncorrelated. In some applications, like motion planning in continuous spaces, similar actions yield similar outcomes. In this paper, we utilize variants of MAB heuristics that make Lipschitz continuity assumptions on the outcomes of actions to improve the efficiency of sampling-based planning approaches. We demonstrate the effectiveness of this approach in the context of motion planning for automated driving.