mlsanityData to 13 Oct 2026

Stochastic Frank-Wolfe Methods for Nonconvex Optimization

arXiv 1607.08254 ↗First posted 27 Jul 2016math.OC cs.LG stat.MLcited by 40

Not on the map. Why?

Abstract (arXiv)

We study Frank-Wolfe methods for nonconvex stochastic and finite-sum optimization problems. Frank-Wolfe methods (in the convex case) have gained tremendous recent interest in machine learning and optimization communities due to their projection-free property and their ability to exploit structured constraints. However, our understanding of these algorithms in the nonconvex setting is fairly limited. In this paper, we propose nonconvex stochastic Frank-Wolfe methods and analyze their convergence properties. For objective functions that decompose into a finite-sum, we leverage ideas from variance reduction techniques for convex optimization to obtain new variance reduced nonconvex Frank-Wolfe methods that have provably faster convergence than the classical Frank-Wolfe method. Finally, we show that the faster convergence rates of our variance reduced methods also translate into improved convergence rates for the stochastic setting.

Key facts. Cited by 40 papers in the map (data through 13 Oct 2026). First posted on arXiv 27 Jul 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 10; 2022: 8; 2023: 4; 2024: 5; 2025: 6; 2026: 7.

40 papers.

Sort:

By area:

  1. 2024Chain of LoRA: Efficient Fine-tuning of Language Models via Residual Learningcited by 81
  2. 2025Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noisecited by 40
  3. 2023Enhancing Sharpness-Aware Optimization Through Variance Suppressioncited by 37
  4. 2023Scalable Optimal Transport Methods in Machine Learning: A Contemporary Surveycited by 27
  5. 2021A Stochastic Sequential Quadratic Optimization Algorithm for Nonlinear Equality Constrained Optimization with Rank-Deficient Jacobianscited by 21
  6. 2021Complexity of Linear Minimization and Projection on Some Setscited by 19
  7. 2021Breaking the Linear Iteration Cost Barrier for Some Well-known Conditional Gradient Methods Using MaxIP Data-structurescited by 19
  8. 2021Escaping from Zero Gradient: Revisiting Action-Constrained Reinforcement Learning via Frank-Wolfe Policy Optimizationcited by 18
  9. 2022Compression-aware Training of Neural Networks using Frank-Wolfecited by 16
  10. 2022Efficient Approximation of Gromov-Wasserstein Distance Using Importance Sparsificationcited by 11
  11. 2021Projection-Free Algorithm for Stochastic Bi-level Optimizationcited by 8
  12. 2022A Projection-free Algorithm for Constrained Stochastic Multi-level Composition Optimizationcited by 8
28 more
  1. 2022CrAM: A Compression-Aware Minimizercited by 7
  2. 2021Zeroth and First Order Stochastic Frank-Wolfe Algorithms for Constrained Optimizationcited by 6
  3. 2022Constrained Stochastic Nonconvex Optimization with State-dependent Markov Datacited by 6
  4. 2023Sarah Frank-Wolfe: Methods for Constrained Optimization with Best Rates and Practical Featurescited by 6
  5. 2021Interpretable Neural Networks with Frank-Wolfe: Sparse Relevance Maps and Relevance Orderingscited by 5
  6. 2021Distributionally Robust Optimization with Markovian Datacited by 4
  7. 2021Shift of Pairwise Similarities for Data Clusteringcited by 4
  8. 2023Optimal Exploration is no harder than Thompson Samplingcited by 3
  9. 2022Variance-Reduced Conservative Policy Iterationcited by 2
  10. 2024EsaCL: Efficient Continual Learning of Sparse Modelscited by 2
  11. 2024Projection-Free Variance Reduction Methods for Stochastic Constrained Multi-Level Compositional Optimizationcited by 2
  12. 2024A General Online Algorithm for Optimizing Complex Performance Metricscited by 2
  13. 2026BLADE: Scalable Bi-level Adaptive Data Selection for LLM Trainingcited by 2
  14. 2022Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimizationcited by 1
  15. 2025An Efficient Local Search Approach for Polarized Community Discovery in Signed Networkscited by 1
  16. 2025VASSO: Variance Suppression for Sharpness-Aware Minimizationcited by 1
  17. 2025Don't Be Greedy, Just Relax! Pruning LLMs via Frank-Wolfecited by 1
  18. 2025Iterative Sampling Methods for Sinkhorn Distributionally Robust Optimizationcited by 1
  19. 2026Adaptive Lipschitz-Free Conditional Gradient Methods for Stochastic Composite Nonconvex Optimizationcited by 1
  20. 2026Policy Regret for Embedding Model Routing: Contextual Bandits with Low-Rank Expertscited by 1
  21. 2021Online Orthogonal Dictionary Learning Based on Frank-Wolfe Methodcited by 0
  22. 2022Using Taylor-Approximated Gradients to Improve the Frank-Wolfe Method for Empirical Risk Minimizationcited by 0
  23. 2024Federated Frank-Wolfe Algorithmcited by 0
  24. 2025Quantitative Convergence Analysis of Projected Stochastic Gradient Descent for Non-Convex Losses via the Goldstein Subdifferentialcited by 0
  25. 2026CLion: Efficient Cautious Lion Optimizer with Enhanced Generalizationcited by 0
  26. 2026Accelerating LMO-Based Optimization via Implicit Gradient Transportcited by 0
  27. 2026Lloyd's K-Means Clustering Algorithm Is Frank-Wolfe in Disguisecited by 0
  28. 2026Variance-Reduced Conditional Gradient Methods under Markovian Sampling for Nonconvex Composite Optimizationcited by 0