mlsanityData to 7 Oct 2026

Efficient displacement convex optimization with particle gradient descent

arXiv 2302.04753 ↗First posted 9 Feb 2023cs.LG stat.MLcited by 5

Not on the map. Why?

Abstract (arXiv)

Particle gradient descent, which uses particles to represent a probability measure and performs gradient descent on particles in parallel, is widely used to optimize functions of probability measures. This paper considers particle gradient descent with a finite number of particles and establishes its theoretical guarantees to optimize functions that are displacement convex in measures. Concretely, for Lipschitz displacement convex functions defined on probability over ℝd, we prove that O(1/ε2) particles and O(d/ε4) computations are sufficient to find the ε-optimal solutions. We further provide improved complexity bounds for optimizing smooth displacement convex functions. We demonstrate the application of our results for function approximation with specific neural architectures with two-dimensional inputs.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 22 of its 57 references that are in the map. First posted on arXiv 9 Feb 2023.

Builds on

22 of 57 references.

References · 22

  1. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  2. 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 422
  3. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 329
  4. 2021The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networkscited by 285
  5. 2017How to Escape Saddle Points Efficientlycited by 284
  6. 2020Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Losscited by 267
  7. 2015Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decompositioncited by 241
  8. 2017No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysiscited by 127
  9. 2016First-order Methods for Geodesically Convex Optimizationcited by 92
  10. 2017Gradient Descent Can Take Exponential Time to Escape Saddle Pointscited by 74
  11. 2017A Hitting Time Analysis of Stochastic Gradient Langevin Dynamicscited by 70
  12. 2018Escaping Saddles with Stochastic Gradientscited by 66
10 more
  1. 2020A Non-Asymptotic Analysis for Stein Variational Gradient Descentcited by 61
  2. 2021Kernel Stein Discrepancy Descentcited by 57
  3. 2017Stochastic Particle Gradient Descent for Infinite Ensemblescited by 52
  4. 2019Sparse Optimization on Measures with Over-parameterized Gradient Descentcited by 50
  5. 2020Convex Geometry and Duality of Over-parameterized Neural Networkscited by 49
  6. 2022Fast Convex Optimization for Two-Layer ReLU Networks: Equivalent Model Classes and Cone Decompositionscited by 31
  7. 2017First-order Stochastic Algorithms for Escaping From Saddle Points in Almost Linear Timecited by 29
  8. 2019Analysis of a Two-Layer Neural Network via Displacement Convexitycited by 27
  9. 2022Sampling with Mollified Interaction Energy Descentcited by 17
  10. 2020Variational Transport: A Convergent Particle-BasedAlgorithm for Distributional Optimizationcited by 6

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 1.

5 papers.

  1. 2023Posterior Sampling Based on Gradient Flows of the MMD with Negative Distance Kernelcited by 25
  2. 2023On the impact of activation and normalization in obtaining isometric embeddings at initializationcited by 13
  3. 2024Wasserstein Gradient Flows of MMD Functionals with Distance Kernel and Cauchy Problems on Quantile Functionscited by 7
  4. 2024C^*-Algebraic Machine Learning: Moving in a New Directioncited by 1
  5. 2025Data Generation without Function Estimationcited by 1