mlsanityData to 7 Oct 2026

Optimal Neural Network Approximation of Wasserstein Gradient Direction via Convex Optimization

arXiv 2205.13098 ↗First posted 26 May 2022cs.LG math.OC stat.MLcited by 9

Not on the map. Why?

Abstract (arXiv)

The computation of Wasserstein gradient direction is essential for posterior sampling problems and scientific computing. The approximation of the Wasserstein gradient with finite samples requires solving a variational problem. We study the variational problem in the family of two-layer networks with squared-ReLU activations, towards which we derive a semi-definite programming (SDP) relaxation. This SDP can be viewed as an approximation of the Wasserstein gradient in a broader function family including two-layer networks. By solving the convex SDP, we obtain the optimal approximation of the Wasserstein gradient direction in this class of functions. Numerical experiments including PDE-constrained Bayesian inference and parameter estimation in COVID-19 modeling demonstrate the effectiveness of the proposed method.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 30 of its 44 references that are in the map. First posted on arXiv 26 May 2022.

Builds on

30 of 44 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Variational Inference with Normalizing Flowscited by 2,161
  3. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 633
  4. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  5. 2017The Expressive Power of Neural Networks: A View from the Widthcited by 318
  6. 2020OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transportcited by 143
  7. 2020Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networkscited by 96
  8. 2021Proximal Optimal Transport Modeling of Population Dynamicscited by 91
  9. 2018Sliced-Wasserstein Flows: Nonparametric Generative Modeling via Optimal Transport and Diffusionscited by 77
  10. 2021Large-Scale Wasserstein Gradient Flowscited by 76
  11. 2021Optimizing Functionals on the Space of Probabilities with Input Convex Neural Networkscited by 53
  12. 2021Variational Wasserstein gradient flowcited by 53
18 more
  1. 2018A Stein variational Newton methodcited by 49
  2. 2014Dimension-independent likelihood-informed MCMCcited by 36
  3. 2020Alternating the Population and Control Neural Networks to Solve High-Dimensional Stochastic Mean-Field Gamescited by 35
  4. 2020Projected Stein Variational Gradient Descentcited by 33
  5. 2020Vector-output ReLU Neural Network Problems are Copositive Programs: Convex Analysis of Two Layer Networks and Polynomial-time Algorithmscited by 33
  6. 2019HINT: Hierarchical Invertible Neural Transport for Density Estimation and Bayesian Inferencecited by 31
  7. 2018Certified dimension reduction in nonlinear Bayesian inverse problemscited by 26
  8. 2021Projected Wasserstein gradient descent for high-dimensional Bayesian inferencecited by 24
  9. 2014SDPNAL+: A Majorized Semismooth Newton-CG Augmented Lagrangian Method for Semidefinite Programming with Nonnegative Constraintscited by 22
  10. 2021Demystifying Batch Normalization in ReLU Networks: Equivalent Convex Optimization Models and Implicit Regularizationcited by 21
  11. 2021Neural Variational Gradient Descentcited by 19
  12. 2021Wasserstein Proximal of GANscited by 15
  13. 2021The Deep Minimizing Movement Schemecited by 15
  14. 2021Neural Spectrahedra and Semidefinite Lifts: Global Convex Optimization of Polynomial Activation Neural Networks in Fully Polynomial-Timecited by 14
  15. 2018Approximate inference with Wasserstein gradient flowscited by 12
  16. 2021Scaling Up Bayesian Uncertainty Quantification for Inverse Problems using Deep Neural Networkscited by 9
  17. 2021Relative Entropy Gradient Sampler for Unnormalized Distributionscited by 5
  18. 2013Trust--Region Problems with Linear Inequality Constraints: Exact SDP Relaxation, Global Optimality and Robust Optimizationcited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 4; 2024: 4.

9 papers.

  1. 2022Particle-based Variational Inference with Preconditioned Functional Gradient Flowcited by 20
  2. 2023Sampling via Gradient Flows in the Space of Probability Measurescited by 20
  3. 2023Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariancecited by 16
  4. 2024Mirror and Preconditioned Gradient Descent in Wasserstein Spacecited by 11
  5. 2023Particle-based Variational Inference with Generalized Wasserstein Gradient Flowcited by 10
  6. 2024Score-based Neural Ordinary Differential Equations for Computing Mean Field Control Problemscited by 7
  7. 2024A score-based particle method for homogeneous Landau equationcited by 6
  8. 2023Leveraging Optimal Transport via Projections on Subspaces for Machine Learning Applicationscited by 1
  9. 2024A Library of Mirrors: Deep Neural Nets in Low Dimensions are Convex Lasso Models with Reflection Featurescited by 1