A Stochastic Variance Reduced Gradient using Barzilai-Borwein Techniques as Second Order Information
Not on the map. Why?
Abstract (arXiv)
In this paper, we consider to improve the stochastic variance reduce gradient (SVRG) method via incorporating the curvature information of the objective function. We propose to reduce the variance of stochastic gradients using the computationally efficient Barzilai-Borwein (BB) method by incorporating it into the SVRG. We also incorporate a BB-step size as its variant. We prove its linear convergence theorem that works not only for the proposed method but also for the other existing variants of SVRG with second-order information. We conduct the numerical experiments on the benchmark datasets and show that the proposed method with constant step size performs better than the existing variance reduced methods for some test problems.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 21 references that are in the map. First posted on arXiv 23 Aug 2022.
Builds on
8 of 21 references.
References · 8
- 2014SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectivescited by 402
- 2013Minimizing Finite Sums with the Stochastic Average Gradientcited by 164
- 2014A Proximal Stochastic Gradient Method with Progressive Variance Reductioncited by 78
- 2014A Stochastic Quasi-Newton Method for Large-Scale Optimizationcited by 69
- 2015A Linearly-Convergent Stochastic L-BFGS Algorithmcited by 39
- 2016Barzilai-Borwein Step Size for Stochastic Gradient Descentcited by 33
- 2017Tracking the gradients using the Hessian: A new look at variance reducing stochastic methodscited by 5
- 2017Stochastic Non-convex Ordinal Embedding with Stabilized Barzilai-Borwein Step Sizecited by 4
Built on by
Not cited yet.