mlsanityData to 14 Oct 2026

Sparse Optimization on Measures with Over-parameterized Gradient Descent

arXiv 1907.10300 ↗First posted 24 Jul 2019math.OC stat.MLcited by 50

Not on the map. Why?

Abstract (arXiv)

Minimizing a convex function of a measure with a sparsity-inducing penalty is a typical problem arising, e.g., in sparse spikes deconvolution or two-layer neural networks training. We show that this problem can be solved by discretizing the measure and running non-convex gradient descent on the positions and weights of the particles. For measures on a d-dimensional manifold and under some non-degeneracy assumptions, this leads to a global optimization algorithm with a complexity scaling as log(1/ε) in the desired accuracy ε, instead of ε-d for convex methods. The key theoretical tools are a local convergence analysis in Wasserstein space and an analysis of a perturbed mirror descent in the space of measures. Our bounds involve quantities that are exponential in d which is unavoidable under our assumptions.

Key facts. Cited by 50 papers in the map (data through 14 Oct 2026). First posted on arXiv 24 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 11; 2022: 14; 2023: 10; 2024: 6; 2025: 4; 2026: 5.

50 papers.

Sort:

By area:

  1. 2022Convex Analysis of the Mean Field Langevin Dynamicscited by 55
  2. 2021A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Networkcited by 47
  3. 2022Unbalanced Optimal Transport, from Theory to Numericscited by 46
  4. 2022Learning sparse features can lead to overfitting in neural networkscited by 23
  5. 2021Understanding Deflation Process in Over-parametrized Tensor Decompositioncited by 20
  6. 2022Sampling with Mollified Interaction Energy Descentcited by 17
  7. 2021Global Convergence of Three-layer Neural Networks in the Mean Field Regimecited by 16
  8. 2021On Learnability via Gradient Method for Two-Layer ReLU Neural Networks in Teacher-Student Settingcited by 16
  9. 2021Gradient Descent on Infinitely Wide Neural Networks: Global Convergence and Generalizationcited by 16
  10. 2023An Explicit Expansion of the Kullback-Leibler Divergence along its Fisher-Rao Gradient Flowcited by 16
  11. 2021Analysis of feature learning in weight-tied autoencoders via the mean field lenscited by 14
  12. 2022An Exponentially Converging Particle Method for the Mixed Nash Equilibrium of Continuous Gamescited by 14
38 more
  1. 2022On Feature Learning in Neural Networks with Global Convergence Guaranteescited by 13
  2. 2024Features are fate: a theory of transfer learning in high-dimensional regressioncited by 13
  3. 2025Weighted quantization using MMD: From mean field to mean shift via gradient flowscited by 12
  4. 2021Convergence analysis for gradient flows in the training of artificial neural networks with ReLU activationcited by 11
  5. 2022Provable Acceleration of Heavy Ball beyond Quadratics for a Class of Polyak-Lojasiewicz Functions when the Non-Convexity is Averaged-Outcited by 11
  6. 2022Excess Risk of Two-Layer ReLU Neural Networks in Teacher-Student Settings and its Superiority to Kernel Methodscited by 10
  7. 2021Existence, uniqueness, and convergence rates for gradient flows in the training of artificial neural networks with ReLU activationcited by 9
  8. 2021Parallel Deep Neural Networks Have Zero Duality Gapcited by 9
  9. 2024Non-convergence to global minimizers for Adam and stochastic gradient descent optimization and constructions of local minimizers in the training of artificial neural networkscited by 9
  10. 2024Interaction-Force Transport Gradient Flowscited by 9
  11. 2022On the symmetries in the dynamics of wide two-layer neural networkscited by 8
  12. 2024Privacy-preserving data release leveraging optimal transport and particle gradient descentcited by 8
  13. 2025Hellinger-Kantorovich Gradient Flows: Global Exponential Decay of Entropy Functionalscited by 7
  14. 2025Understanding the Mechanisms of Fast Hyperparameter Transfercited by 7
  15. 2023Rethinking Gauss-Newton for learning over-parameterized modelscited by 6
  16. 2023Continuized Acceleration for Quasar Convex Functions in Non-Convex Optimizationcited by 6
  17. 2021Limiting fluctuation and trajectorial stability of multilayer neural networks with mean field trainingcited by 5
  18. 2023Convergence beyond the over-parameterized regime using Rayleigh quotientscited by 5
  19. 2023Efficient displacement convex optimization with particle gradient descentcited by 5
  20. 2023A convergence result of a continuous model of deep learning via a Lojasiewicz--Simon inequalitycited by 5
  21. 2022A Functional-Space Mean-Field Theory of Partially-Trained Three-Layer Neural Networkscited by 3
  22. 2023FastPart: Over-Parameterized Stochastic Gradient Descent for Sparse optimisation on Measurescited by 3
  23. 2026Uniform-in-Time Weak Propagation-of-Chaos in Shallow Neural Networkscited by 3
  24. 2022Off-the-grid learning of mixtures from a continuous dictionarycited by 2
  25. 2024Second Maximum of a Gaussian Random Field and Exact (t-)Spacing testcited by 2
  26. 2024Nonuniform random feature models using derivative informationcited by 2
  27. 2021WGAN with an Infinitely Wide Generator Has No Spurious Stationary Pointscited by 1
  28. 2022Polynomial-time Sparse Measure Recovery: From Mean Field Theory to Algorithm Designcited by 1
  29. 2023Global Optimality of Elman-type RNN in the Mean-Field Regimecited by 1
  30. 2023Fixing the NTK: From Neural Network Linearizations to Exact Convex Programscited by 1
  31. 2026Fenchel-Young Duality Gaps: Certified Early Stopping for Regularized Inverse Problemscited by 1
  32. 2022Simultaneous off-the-grid learning of mixtures issued from a continuous dictionarycited by 0
  33. 2022Proximal methods for point source localisationcited by 0
  34. 2023Towards Optimal Randomized Strategies in Adversarial Example Gamecited by 0
  35. 2025Effective regions and kernels in continuous sparse regularisation, with application to sketched mixturescited by 0
  36. 2026Sinkhorn Based Associative Memory Retrieval Using Spherical Hellinger Kantorovich Dynamicscited by 0
  37. 2026Singular perturbations and hierarchical learning in two-layer neural networkscited by 0
  38. 2026Riemannian Gradient Descent for Gaussian Mixture Models with unknown diagonal covariancescited by 0