Approximating Langevin Monte Carlo with ResNet-like Neural Network architectures
Not on the map. Why?
Abstract (arXiv)
We sample from a given target distribution by constructing a neural network which maps samples from a simple reference, e.g. the standard normal distribution, to samples from the target. To that end, we propose using a neural network architecture inspired by the Langevin Monte Carlo (LMC) algorithm. Based on LMC perturbation results, we show approximation rates of the proposed architecture for smooth, log-concave target distributions measured in the Wasserstein-2 distance. The analysis heavily relies on the notion of sub-Gaussianity of the intermediate measures of the perturbed LMC process. In particular, we derive bounds on the growth of the intermediate variance proxies under different assumptions on the perturbations. Moreover, we propose an architecture similar to deep residual neural networks and derive expressivity results for approximating the sample to target distribution map.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 44 of its 86 references that are in the map. First posted on arXiv 6 Nov 2023.
Builds on
44 of 86 references.
References · 44
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2013Auto-Encoding Variational Bayescited by 4,053
- 2018Neural Ordinary Differential Equationscited by 3,779
- 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
- 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
- 2016Density estimation using Real NVPcited by 2,042
- 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
- 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 640
- 2016Error bounds for approximations with deep ReLU networkscited by 574
- 2021Physics-informed neural networks (PINNs) for fluid mechanics: A reviewcited by 473
32 more
- 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 331
- 2016NIPS 2016 Tutorial: Generative Adversarial Networkscited by 285
- 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 252
- 2017Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysiscited by 243
- 2018Analyzing Inverse Problems with Invertible Neural Networkscited by 242
- 2014Theoretical guarantees for approximate sampling from smooth and log-concave densitiescited by 208
- 2015Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithmcited by 197
- 2017Optimal approximation of piecewise smooth functions using deep ReLU neural networkscited by 179
- 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 166
- 2016High-dimensional Bayesian inference via the Unadjusted Langevin Algorithmcited by 146
- 2018Log-concave sampling: Metropolis-Hastings algorithms are fastcited by 118
- 2017User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradientcited by 116
- 2021An Introduction to Deep Generative Modelingcited by 104
- 2017Optimal Approximation with Sparsely Connected Deep Neural Networkscited by 91
- 2017Convergence of Langevin MCMC in KL-divergencecited by 88
- 2019The phase diagram of approximation rates for deep neural networkscited by 88
- 2019A Theoretical Analysis of Deep Neural Networks and Parametric PDEscited by 74
- 2020Faster Wasserstein Distance Estimation with the Sinkhorn Divergencecited by 73
- 2018Sharp convergence rates for Langevin dynamics in the nonconvex settingcited by 68
- 2020Neural Network Approximation: Three Hidden Layers Are Enoughcited by 63
- 2020Wasserstein Control of Mirror Langevin Monte Carlocited by 55
- 2019Approximation spaces of deep neural networkscited by 51
- 2019Error bounds for deep ReLU networks using the Kolmogorov--Arnold superposition theoremcited by 49
- 2021Exponential Convergence of Deep Operator Networks for Elliptic Partial Differential Equationscited by 44
- 2019Nonasymptotic estimates for Stochastic Gradient Langevin Dynamics under local conditions in nonconvex optimizationcited by 38
- 2018Non-asymptotic bounds for sampling algorithms without log-concavitycited by 35
- 2018A proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficientscited by 31
- 2020Numerical Solution of the Parametric Diffusion Equation by Deep Neural Networkscited by 27
- 2022Do Residual Neural Networks discretize Neural Ordinary Differential Equations?cited by 27
- 2018The universal approximation power of finite-width deep ReLU networkscited by 12
- 2023Multilevel CNNs for Parametric PDEscited by 9
- 2022Concentration of the Langevin Algorithm's Stationary Distributioncited by 5
Built on by
Citing papers per year: 2024: 2.
2 papers.