mlsanityData to 7 Oct 2026

FastPart: Over-Parameterized Stochastic Gradient Descent for Sparse optimisation on Measures

arXiv 2312.05993 ↗First posted 10 Dec 2023math.OC stat.MLcited by 3

Not on the map. Why?

Abstract (arXiv)

This paper presents a novel algorithm that leverages Stochastic Gradient Descent strategies in conjunction with Random Features to augment the scalability of Conic Particle Gradient Descent (CPGD) specifically tailored for solving sparse optimization problems on measures. By formulating the CPGD steps within a variational framework, we provide rigorous mathematical proofs demonstrating the following key findings: (i) The total variation norms of the solution measures along the descent trajectory remain bounded, ensuring stability and preventing undesirable divergence; (ii) We establish a global convergence guarantee with a convergence rate of O(log(K)/√K) over K iterations, showcasing the efficiency and effectiveness of our algorithm, (iii) Additionally, we analyse and establish local control over the first-order condition discrepancy, contributing to a deeper understanding of the algorithm's behaviour and reliability in practical applications.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 13 of its 35 references that are in the map. First posted on arXiv 10 Dec 2023.

Builds on

13 of 35 references.

References · 13

  1. 2014Convex Optimization: Algorithms and Complexitycited by 569
  2. 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 422
  3. 2016Kernel Mean Embedding of Distributions: A Review and Beyondcited by 360
  4. 2013Mini-batch Stochastic Approximation Methods for Nonconvex Stochastic Composite Optimizationcited by 142
  5. 2019Sparse Optimization on Measures with Over-parameterized Gradient Descentcited by 50
  6. 2018On Representer Theorems and Convex Regularizationcited by 24
  7. 2013Exact Support Recovery for Sparse Spikes Deconvolutioncited by 22
  8. 2020Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descentcited by 18
  9. 2018The Sliding Frank-Wolfe Algorithm and its Application to Super-Resolution Microscopycited by 16
  10. 2021Gradient Descent on Infinitely Wide Neural Networks: Global Convergence and Generalizationcited by 15
  11. 2016Sketching for Large-Scale Learning of Mixture Modelscited by 14
  12. 2013Support detection in super-resolutioncited by 2
1 more
  1. 2021Asymptotic linear convergence of fully-corrective generalized conditional gradient methodscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026Fenchel-Young Duality Gaps: Certified Early Stopping for Regularized Inverse Problemscited by 1
  2. 2025Effective regions and kernels in continuous sparse regularisation, with application to sketched mixturescited by 0
  3. 2026Riemannian Gradient Descent for Gaussian Mixture Models with unknown diagonal covariancescited by 0