# Reducing normalizing flow complexity for MCMC preconditioning

arXiv: https://arxiv.org/abs/2511.02345 · HTML: https://mlsanity.com/paper/2511.02345/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 43 references that are in the map.
- First posted on arXiv 4 Nov 2025.

## Builds on (most cited first, 22 of 22)

- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro](https://mlsanity.com/paper/1912.11554/) (2019)
- [Causal Autoregressive Flows](https://mlsanity.com/paper/2011.02268/) (2020)
- [Annealed Flow Transport Monte Carlo](https://mlsanity.com/paper/2102.07501/) (2021)
- [NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transport](https://mlsanity.com/paper/1903.03704/) (2019)
- [Continual Repeated Annealed Flow Transport Monte Carlo](https://mlsanity.com/paper/2201.13117/) (2022)
- [Hamiltonian Monte Carlo for Hierarchical Models](https://mlsanity.com/paper/1312.0906/) (2013)
- [Causal normalizing flows: from theory to practice](https://mlsanity.com/paper/2306.05415/) (2023)
- [Graphical Normalizing Flows](https://mlsanity.com/paper/2006.02548/) (2020)
- [BlackJAX: Composable Bayesian inference in JAX](https://mlsanity.com/paper/2402.10797/) (2024)
- [Local-Global MCMC kernels: the best of both worlds](https://mlsanity.com/paper/2111.02702/) (2021)
- [On Sampling with Approximate Transport Maps](https://mlsanity.com/paper/2302.04763/) (2023)
- [Automatic Reparameterisation of Probabilistic Programs](https://mlsanity.com/paper/1906.03028/) (2019)
- [Efficient Bayesian Sampling Using Normalizing Flows to Assist Markov Chain Monte Carlo Methods](https://mlsanity.com/paper/2107.08001/) (2021)
- [Personalized Public Policy Analysis in Social Sciences using Causal-Graphical Normalizing Flows](https://mlsanity.com/paper/2202.03281/) (2022)
- [Deterministic Langevin Monte Carlo with Normalizing Flows for Bayesian Inference](https://mlsanity.com/paper/2205.14240/) (2022)
- [Graphical Residual Flows](https://mlsanity.com/paper/2204.11846/) (2022)
- [Flow Annealed Kalman Inversion for Gradient-Free Inference in Bayesian Inverse Problems](https://mlsanity.com/paper/2309.11490/) (2023)
- [Empirical evaluation of normalizing flows in Markov Chain Monte Carlo](https://mlsanity.com/paper/2412.17136/) (2024)
- [Escaping Neal's Funnel: a multi-stage sampling method for hierarchical models](https://mlsanity.com/paper/2510.12917/) (2025)

## Abstract (arXiv)

Preconditioning is a key component of MCMC algorithms that improves sampling efficiency by facilitating exploration of geometrically complex target distributions through an invertible map. While linear preconditioners are often sufficient for moderately complex target distributions, recent work has explored nonlinear preconditioning with invertible neural networks as components of normalizing flows (NFs). However, empirical and theoretical studies show that overparameterized NF preconditioners can degrade sampling efficiency and fit quality. Moreover, existing NF-based approaches do not adapt their architectures to the target distribution. Related work outside of MCMC similarly finds that suitably parameterized NFs can achieve comparable or superior performance with substantially less training time or data. We propose a factorized preconditioning architecture that reduces NF complexity by combining a linear component with a conditional NF, improving adaptability to target geometry. The linear preconditioner is applied to dimensions that are approximately Gaussian, as estimated from warmup samples, while the conditional NF models more complex dimensions. Our method yields significantly better tail samples on two complex synthetic distributions and consistently better performance on a sparse logistic regression posterior across varying likelihood and prior strengths. It also achieves higher effective sample sizes on hierarchical Bayesian model posteriors with weak likelihoods and strong funnel geometries. This approach is particularly relevant for hierarchical Bayesian model analyses with limited data and could inform current theoretical and software strides in neural MCMC design.