# Sequential transport maps using SoS density estimation and α-divergences

arXiv: https://arxiv.org/abs/2402.17943 · HTML: https://mlsanity.com/paper/2402.17943/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 57 references that are in the map.
- First posted on arXiv 27 Feb 2024.

## Builds on (most cited first, 27 of 27)

- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Iterated Denoising Energy Matching for Sampling from Boltzmann Densities](https://mlsanity.com/paper/2402.06121/) (2024)
- [An elementary introduction to information geometry](https://mlsanity.com/paper/1808.08271/) (2018)
- [RNADE: The real-valued neural autoregressive density-estimator](https://mlsanity.com/paper/1306.0186/) (2013)
- [Sum-of-Squares Polynomial Flow](https://mlsanity.com/paper/1905.02325/) (2019)
- [Coupling techniques for nonlinear ensemble filtering](https://mlsanity.com/paper/1907.00389/) (2019)
- [Inference via low-dimensional couplings](https://mlsanity.com/paper/1703.06131/) (2017)
- [Non-parametric Models for Non-negative Functions](https://mlsanity.com/paper/2007.03926/) (2020)
- [On the representation and learning of monotone triangular transport maps](https://mlsanity.com/paper/2009.10303/) (2020)
- [JuMP 1.0: Recent improvements to a modeling language for mathematical optimization](https://mlsanity.com/paper/2206.03866/) (2022)
- [Stochastic Localization via Iterative Posterior Sampling](https://mlsanity.com/paper/2402.10758/) (2024)
- [Sampling in Unit Time with Kernel Fisher-Rao Flow](https://mlsanity.com/paper/2401.03892/) (2024)
- [Conditional Optimal Transport on Function Spaces](https://mlsanity.com/paper/2311.05672/) (2023)
- [On Sampling with Approximate Transport Maps](https://mlsanity.com/paper/2302.04763/) (2023)
- [An introduction to sampling via measure transport](https://mlsanity.com/paper/1602.05023/) (2016)
- [Deep composition of tensor-trains using squared inverse Rosenblatt transports](https://mlsanity.com/paper/2007.06968/) (2020)
- [Scalable conditional deep inverse Rosenblatt transports using tensor-trains and gradient-based dimension reduction](https://mlsanity.com/paper/2106.04170/) (2021)
- [Designing losses for data-free training of normalizing flows on Boltzmann distributions](https://mlsanity.com/paper/2301.05475/) (2023)
- [Solving natural conic formulations with Hypatia.jl](https://mlsanity.com/paper/2005.01136/) (2020)
- [Self-reinforced polynomial approximation methods for concentrated probability densities](https://mlsanity.com/paper/2303.02554/) (2023)
- [Deep importance sampling using tensor trains with application to a priori and a posteriori rare event estimation](https://mlsanity.com/paper/2209.01941/) (2022)

## Built on by (most cited first, 1 of 1)

- [A friendly introduction to triangular transport](https://mlsanity.com/paper/2503.21673/) (2025)

## Abstract (arXiv)

Transport-based density estimation methods are receiving growing interest because of their ability to efficiently generate samples from the approximated density. We further invertigate the sequential transport maps framework proposed from arXiv:2106.04170 arXiv:2303.02554, which builds on a sequence of composed Knothe-Rosenblatt (KR) maps. Each of those maps are built by first estimating an intermediate density of moderate complexity, and then by computing the exact KR map from a reference density to the precomputed approximate density. In our work, we explore the use of Sum-of-Squares (SoS) densities and α-divergences for approximating the intermediate densities. Combining SoS densities with α-divergence interestingly yields convex optimization problems which can be efficiently solved using semidefinite programming. The main advantage of α-divergences is to enable working with unnormalized densities, which provides benefits both numerically and theoretically. In particular, we provide a new convergence analyses of the sequential transport maps based on information geometric properties of α-divergences. The choice of intermediate densities is also crucial for the efficiency of the method. While tempered (or annealed) densities are the state-of-the-art, we introduce diffusion-based intermediate densities which permits to approximate densities known from samples only. Such intermediate densities are well-established in machine learning for generative modeling. Finally we propose low-dimensional maps (or lazy maps) for dealing with high-dimensional problems and numerically demonstrate our methods on Bayesian inference problems and unsupervised learning tasks.