# Flow Annealed Importance Sampling Bootstrap

arXiv: https://arxiv.org/abs/2208.01893 · HTML: https://mlsanity.com/paper/2208.01893/

## Key facts

- Cited by 127 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 52 references that are in the map.
- First posted on arXiv 3 Aug 2022.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [Torsional Diffusion for Molecular Conformer Generation](https://mlsanity.com/paper/2206.01729/) (2022)
- [Stochastic Normalizing Flows](https://mlsanity.com/paper/2002.06707/) (2020)
- [Neural Importance Sampling](https://mlsanity.com/paper/1808.03856/) (2018)
- [Normalizing Flows on Tori and Spheres](https://mlsanity.com/paper/2002.02428/) (2020)
- [Annealed Flow Transport Monte Carlo](https://mlsanity.com/paper/2102.07501/) (2021)
- [Gradients are Not All You Need](https://mlsanity.com/paper/2111.05803/) (2021)
- [SurVAE Flows: Surjections to Bridge the Gap between VAEs and Flows](https://mlsanity.com/paper/2007.02731/) (2020)
- [Machine Learning of coarse-grained Molecular Dynamics Force Fields](https://mlsanity.com/paper/1812.01736/) (2018)
- [Continual Repeated Annealed Flow Transport Monte Carlo](https://mlsanity.com/paper/2201.13117/) (2022)
- [Smooth Normalizing Flows](https://mlsanity.com/paper/2110.00351/) (2021)
- [Generalizing Hamiltonian Monte Carlo with Neural Networks](https://mlsanity.com/paper/1711.09268/) (2017)
- [Resampling Base Distributions of Normalizing Flows](https://mlsanity.com/paper/2110.15828/) (2021)
- [MCMC Variational Inference via Uncorrected Hamiltonian Annealing](https://mlsanity.com/paper/2107.04150/) (2021)
- [Normalizing flows for atomic solids](https://mlsanity.com/paper/2111.08696/) (2021)
- [Differentiable Annealed Importance Sampling and the Perils of Gradient Noise](https://mlsanity.com/paper/2107.10211/) (2021)
- [Generalized Doubly Reparameterized Gradient Estimators](https://mlsanity.com/paper/2101.11046/) (2021)
- [Annealed Importance Sampling with q-Paths](https://mlsanity.com/paper/2012.07823/) (2020)
- [Bootstrap Your Flow](https://mlsanity.com/paper/2111.11510/) (2021)

## Built on by (most cited first, 30 of 127)

- [An optimal control perspective on diffusion-based generative modeling](https://mlsanity.com/paper/2211.01364/) (2022)
- [Equivariant flow matching](https://mlsanity.com/paper/2306.15030/) (2023)
- [Iterated Denoising Energy Matching for Sampling from Boltzmann Densities](https://mlsanity.com/paper/2402.06121/) (2024)
- [AlphaFold Meets Flow Matching for Generating Protein Ensembles](https://mlsanity.com/paper/2402.04845/) (2024)
- [Improved sampling via learned diffusions](https://mlsanity.com/paper/2307.01198/) (2023)
- [Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Experts](https://mlsanity.com/paper/2503.02819/) (2025)
- [Adjoint Sampling: Highly Scalable Diffusion Samplers via Adjoint Matching](https://mlsanity.com/paper/2504.11713/) (2025)
- [Efficient Online Reinforcement Learning for Diffusion Policy](https://mlsanity.com/paper/2502.00361/) (2025)
- [NETS: A Non-Equilibrium Transport Sampler](https://mlsanity.com/paper/2410.02711/) (2024)
- [On the Generalization Properties of Diffusion Models](https://mlsanity.com/paper/2311.01797/) (2023)
- [Particle Denoising Diffusion Sampler](https://mlsanity.com/paper/2402.06320/) (2024)
- [Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimization](https://mlsanity.com/paper/2310.02679/) (2023)
- [normflows: A PyTorch Package for Normalizing Flows](https://mlsanity.com/paper/2302.12014/) (2023)
- [Beyond ELBOs: A Large-Scale Evaluation of Variational Methods for Sampling](https://mlsanity.com/paper/2406.07423/) (2024)
- [Generative Modeling of Molecular Dynamics Trajectories](https://mlsanity.com/paper/2409.17808/) (2024)
- [Sequential Controlled Langevin Diffusions](https://mlsanity.com/paper/2412.07081/) (2024)
- [Transferable Boltzmann Generators](https://mlsanity.com/paper/2406.14426/) (2024)
- [Learning Interpolations between Boltzmann Densities](https://mlsanity.com/paper/2301.07388/) (2023)
- [Scalable Equilibrium Sampling with Sequential Boltzmann Generators](https://mlsanity.com/paper/2502.18462/) (2025)
- [SE(3) Equivariant Augmented Coupling Flows](https://mlsanity.com/paper/2308.10364/) (2023)
- [Underdamped Diffusion Bridges with Applications to Sampling](https://mlsanity.com/paper/2503.01006/) (2025)
- [No Trick, No Treat: Pursuits and Challenges Towards Simulation-free Training of Neural Samplers](https://mlsanity.com/paper/2502.06685/) (2025)
- [Rigid Body Flows for Sampling Molecular Crystal Structures](https://mlsanity.com/paper/2301.11355/) (2023)
- [Towards Predicting Equilibrium Distributions for Molecular Systems with Deep Learning](https://mlsanity.com/paper/2306.05445/) (2023)
- [Consistent Sampling and Simulation: Molecular Dynamics with Energy-Based Diffusion Models](https://mlsanity.com/paper/2506.17139/) (2025)
- [Trust Region Constrained Measure Transport in Path Space for Stochastic Optimal Control and Inference](https://mlsanity.com/paper/2508.12511/) (2025)
- [On Sampling with Approximate Transport Maps](https://mlsanity.com/paper/2302.04763/) (2023)
- [Transport meets Variational Inference: Controlled Monte Carlo Diffusions](https://mlsanity.com/paper/2307.01050/) (2023)
- [Diffusive Gibbs Sampling](https://mlsanity.com/paper/2402.03008/) (2024)
- [Training Neural Samplers with Reverse Diffusive KL Divergence](https://mlsanity.com/paper/2410.12456/) (2024)

## Abstract (arXiv)

Normalizing flows are tractable density models that can approximate complicated target distributions, e.g. Boltzmann distributions of physical systems. However, current methods for training flows either suffer from mode-seeking behavior, use samples from the target generated beforehand by expensive MCMC methods, or use stochastic losses that have high variance. To avoid these problems, we augment flows with annealed importance sampling (AIS) and minimize the mass-covering α-divergence with α=2, which minimizes importance weight variance. Our method, Flow AIS Bootstrap (FAB), uses AIS to generate samples in regions where the flow is a poor approximation of the target, facilitating the discovery of new modes. We apply FAB to multimodal targets and show that we can approximate them very accurately where previous methods fail. To the best of our knowledge, we are the first to learn the Boltzmann distribution of the alanine dipeptide molecule using only the unnormalized target density, without access to samples generated via Molecular Dynamics (MD) simulations: FAB produces better results than training via maximum likelihood on MD samples while using 100 times fewer target evaluations. After reweighting the samples, we obtain unbiased histograms of dihedral angles that are almost identical to the ground truth.