# Adaptive importance sampling for control and inference

arXiv: https://arxiv.org/abs/1505.01874 · HTML: https://mlsanity.com/paper/1505.01874/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 May 2015.

## Built on by (most cited first, 23 of 23)

- [Denoising Diffusion Samplers](https://mlsanity.com/paper/2302.13834/) (2023)
- [Annealed Flow Transport Monte Carlo](https://mlsanity.com/paper/2102.07501/) (2021)
- [Generalized Schr"odinger Bridge Matching](https://mlsanity.com/paper/2310.02233/) (2023)
- [Stochastic Optimal Control Matching](https://mlsanity.com/paper/2312.02027/) (2023)
- [Stochastic Optimal Control for Collective Variable Free Sampling of Molecular Transition Paths](https://mlsanity.com/paper/2207.02149/) (2022)
- [Trust Region Constrained Measure Transport in Path Space for Stochastic Optimal Control and Inference](https://mlsanity.com/paper/2508.12511/) (2025)
- [Reinforcement learning of rare diffusive dynamics](https://mlsanity.com/paper/2105.04321/) (2021)
- [Variational Inference MPC using Normalizing Flows and Out-of-Distribution Projection](https://mlsanity.com/paper/2205.04667/) (2022)
- [A Taxonomy of Loss Functions for Stochastic Optimal Control](https://mlsanity.com/paper/2410.00345/) (2024)
- [Harmonic Path Integral Diffusion](https://mlsanity.com/paper/2409.15166/) (2024)
- [Inferring Smooth Control: Monte Carlo Posterior Policy Iteration with Gaussian Processes](https://mlsanity.com/paper/2210.03512/) (2022)
- [Variational Inference for SDEs Driven by Fractional Noise](https://mlsanity.com/paper/2310.12975/) (2023)
- [Combining Reinforcement Learning and Tensor Networks, with an Application to Dynamical Large Deviations](https://mlsanity.com/paper/2209.14089/) (2022)
- [Efficient Stochastic Optimal Control through Approximate Bayesian Input Inference](https://mlsanity.com/paper/2105.07693/) (2021)
- [Generative Stochastic Optimal Transport: Guided Harmonic Path-Integral Diffusion](https://mlsanity.com/paper/2512.11859/) (2025)
- [Global convergence of optimized adaptive importance samplers](https://mlsanity.com/paper/2201.00409/) (2022)
- [Stochastic Optimal Control for Continuous-Time fMRI Representation Learning](https://mlsanity.com/paper/2502.04892/) (2025)
- [Entangled Schr"odinger Bridge Matching](https://mlsanity.com/paper/2511.07406/) (2025)
- [Adaptive Path Integral Diffusion: AdaPID](https://mlsanity.com/paper/2512.11858/) (2025)
- [Analytic Bridge Diffusions for Controlled Path Generation](https://mlsanity.com/paper/2605.02961/) (2026)
- [Probabilistic Control and Majorization of Optimal Control](https://mlsanity.com/paper/2205.03279/) (2022)
- [QMaxCal: Path-Space Regularization for Open Quantum Control via Girsanov's Theorem](https://mlsanity.com/paper/2606.19947/) (2026)
- [Path Integral Value Matching for Linear Quadratic Stochastic Optimal Control](https://mlsanity.com/paper/2608.10777/) (2026)

## Abstract (arXiv)

Path integral (PI) control problems are a restricted class of non-linear control problems that can be solved formally as a Feyman-Kac path integral and can be estimated using Monte Carlo sampling. In this contribution we review path integral control theory in the finite horizon case. We subsequently focus on the problem how to compute and represent control solutions. Within the PI theory, the question of how to compute becomes the question of importance sampling. Efficient importance samplers are state feedback controllers and the use of these requires an efficient representation. Learning and representing effective state-feedback controllers for non-linear stochastic control problems is a very challenging, and largely unsolved, problem. We show how to learn and represent such controllers using ideas from the cross entropy method. We derive a gradient descent method that allows to learn feed-back controllers using an arbitrary parametrisation. We refer to this method as the Path Integral Cross Entropy method or PICE. We illustrate this method for some simple examples. The path integral control methods can be used to estimate the posterior distribution in latent state models. In neuroscience these problems arise when estimating connectivity from neural recording data using EM. We demonstrate the path integral control method as an accurate alternative to particle filtering.