# Preconditioned Regularized Wasserstein Proximal Sampling

arXiv: https://arxiv.org/abs/2509.01685 · HTML: https://mlsanity.com/paper/2509.01685/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 46 references that are in the map.
- First posted on arXiv 1 Sep 2025.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [Limitations of the Empirical Fisher Approximation for Natural Gradient Descent](https://mlsanity.com/paper/1905.12558/) (2019)
- [VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning](https://mlsanity.com/paper/1705.07761/) (2017)
- [A mathematical perspective on Transformers](https://mlsanity.com/paper/2312.10794/) (2023)
- [High-dimensional Bayesian inference via the Unadjusted Langevin Algorithm](https://mlsanity.com/paper/1605.01559/) (2016)
- [Sinkformers: Transformers with Doubly Stochastic Attention](https://mlsanity.com/paper/2110.11773/) (2021)
- [Bayesian imaging using Plug & Play priors: when Langevin meets Tweedie](https://mlsanity.com/paper/2103.04715/) (2021)
- [Efficient Bayesian computation by proximal Markov chain Monte Carlo: when Langevin meets Moreau](https://mlsanity.com/paper/1612.07471/) (2016)
- [Mirrored Langevin Dynamics](https://mlsanity.com/paper/1802.10174/) (2018)
- [Accelerated Information Gradient flow](https://mlsanity.com/paper/1909.02102/) (2019)
- [A Unified Perspective on the Dynamics of Deep Transformers](https://mlsanity.com/paper/2501.18322/) (2025)
- [DeepInverse: A Python package for solving imaging inverse problems with deep learning](https://mlsanity.com/paper/2505.20160/) (2025)
- [FAdam: Adam is a natural gradient optimizer using diagonal empirical Fisher information](https://mlsanity.com/paper/2405.12807/) (2024)
- [A kernel formula for regularized Wasserstein proximal operators](https://mlsanity.com/paper/2301.10301/) (2023)
- [Subgradient Langevin Methods for Sampling from Non-smooth Potentials](https://mlsanity.com/paper/2308.01417/) (2023)
- [Noise-Free Sampling Algorithms via Regularized Wasserstein Proximals](https://mlsanity.com/paper/2308.14945/) (2023)
- [Laplace Meets Moreau: Smooth Approximation to Infimal Convolutions Using Laplace's Method](https://mlsanity.com/paper/2406.02003/) (2024)
- [Diffusion at Absolute Zero: Langevin Sampling Using Successive Moreau Envelopes [conference paper]](https://mlsanity.com/paper/2502.01358/) (2025)
- [Tensor train based sampling algorithms for approximating regularized Wasserstein proximal operators](https://mlsanity.com/paper/2401.13125/) (2024)
- [Unsupervised Training of Convex Regularizers using Maximum Likelihood Estimation](https://mlsanity.com/paper/2404.05445/) (2024)

## Built on by (most cited first, 1 of 1)

- [Accelerated Regularized Wasserstein Proximal Sampling Algorithms](https://mlsanity.com/paper/2601.09848/) (2026)

## Abstract (arXiv)

We consider sampling from a Gibbs distribution by evolving finitely many particles. We propose a preconditioned version of a recently proposed noise-free sampling method, governed by approximating the score function with the numerically tractable score of a regularized Wasserstein proximal operator. This is derived by a Cole--Hopf transformation on coupled anisotropic heat equations, yielding a kernel formulation for the preconditioned regularized Wasserstein proximal. The diffusion component of the proposed method is also interpreted as a modified self-attention block, as in transformer architectures. For quadratic potentials, we provide a discrete-time non-asymptotic convergence analysis and explicitly characterize the bias, which is dependent on regularization and independent of step-size. Experiments demonstrate acceleration and particle-level stability on various log-concave and non-log-concave toy examples to Bayesian total-variation regularized image deconvolution, and competitive/better performance on non-convex Bayesian neural network training when utilizing variable preconditioning matrices.