mlsanityData to 14 Oct 2026

SARAH-based Variance-reduced Algorithm for Stochastic Finite-sum Cocoercive Variational Inequalities

arXiv 2210.05994 ↗First posted 12 Oct 2022math.OC cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Variational inequalities are a broad formalism that encompasses a vast number of applications. Motivated by applications in machine learning and beyond, stochastic methods are of great importance. In this paper we consider the problem of stochastic finite-sum cocoercive variational inequalities. For this class of problems, we investigate the convergence of the method based on the SARAH variance reduction technique. We show that for strongly monotone problems it is possible to achieve linear convergence to a solution using this method. Experiments confirm the importance and practical applicability of our approach.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 41 references that are in the map. First posted on arXiv 12 Oct 2022.

Builds on

32 of 41 references.

References · 32

  1. 2022Generative Adversarial Networkscited by 5,627
  2. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
  3. 2014SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectivescited by 402
  4. 2019Momentum-Based Variance Reduction in Non-Convex SGDcited by 306
  5. 2018SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimatorcited by 287
  6. 2019FreeLB: Enhanced Adversarial Training for Natural Language Understandingcited by 264
  7. 2017SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradientcited by 220
  8. 2013Minimizing Finite Sums with the Stochastic Average Gradientcited by 164
  9. 2020Better Theory for SGD in the Nonconvex Worldcited by 158
  10. 2018A Variational Inequality Perspective on Generative Adversarial Networkscited by 144
  11. 2020PAGE: A Simple and Optimal Probabilistic Gradient Estimator for Nonconvex Optimizationcited by 119
  12. 2020Adversarial Training for Large Neural Language Modelscited by 111
20 more
  1. 2019On the convergence of single-call stochastic extra-gradient methodscited by 89
  2. 2019A Unified Theory of SGD: Variance Reduction, Sampling, Quantization and Coordinate Descentcited by 84
  3. 2019Reducing Noise in GAN Training with Variance Reduced Extragradientcited by 62
  4. 2016Stochastic Variance Reduction Methods for Saddle-Point Problemscited by 60
  5. 2020Explore Aggressively, Update Conservatively: Stochastic Extragradient Methods with Variable Stepsize Scalingcited by 49
  6. 2021Stochastic Variance Reduction for Variational Inequality Methodscited by 47
  7. 2021Stochastic Gradient Descent-Ascent and Consensus Optimization for Smooth Games: Convergence Analysis under Expected Co-coercivitycited by 43
  8. 2015Improved SVRG for Non-Strongly-Convex or Sum-of-Non-Convex Objectivescited by 38
  9. 2022Stochastic Gradient Descent-Ascent: Unified Theory and New Efficient Methodscited by 37
  10. 2014Incremental Majorization-Minimization Optimization with Application to Large-Scale Machine Learningcited by 35
  11. 2019Revisiting Stochastic Extragradientcited by 35
  12. 2021Stochastic Extragradient: General Analysis and Improved Ratescited by 31
  13. 2018New Convergence Aspects of Stochastic Gradient Algorithmscited by 30
  14. 2017Stochastic Recursive Gradient Algorithm for Nonconvex Optimizationcited by 20
  15. 2020Distributed Saddle-Point Problems: Lower Bounds, Near-Optimal and Robust Algorithmscited by 14
  16. 2018Inexact SARAH Algorithm for Stochastic Optimizationcited by 12
  17. 2019SAGA with Arbitrary Samplingcited by 10
  18. 2022Optimal Gradient Sliding and its Application to Distributed Optimization Under Similaritycited by 10
  19. 2019Accelerating Mini-batch SARAH by Step Size Rulescited by 4
  20. 2022A Unified Analysis of Variational Inequality Methods: Variance Reduction, Sampling, Quantization and Coordinate Descentcited by 4

Built on by

Not cited yet.