mlsanityData to 7 Oct 2026

On Acceleration with Noise-Corrupted Gradients

arXiv 1805.12591 ↗First posted 31 May 2018math.OC cs.DScited by 20

Not on the map. Why?

Abstract (arXiv)

Accelerated algorithms have broad applications in large-scale optimization, due to their generality and fast convergence. However, their stability in the practical setting of noise-corrupted gradient oracles is not well-understood. This paper provides two main technical contributions: (i) a new accelerated method AGDP that generalizes Nesterov's AGD and improves on the recent method AXGD (Diakonikolas & Orecchia, 2018), and (ii) a theoretical study of accelerated algorithms under noisy and inexact gradient oracles, which is supported by numerical experiments. This study leverages the simplicity of AGDP and its analysis to clarify the interaction between noise and acceleration and to suggest modifications to the algorithm that reduce the mean and variance of the error incurred due to the gradient noise.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 12; 2022: 3; 2023: 2; 2024: 3.

20 papers.

Sort:

  1. 2021Stochastic Bias-Reduced Gradient Methodscited by 25
  2. 2022Reproducibility in Optimization: Theoretical Framework and Limitscited by 21
  3. 2021Distributed Learning Systems with First-order Methodscited by 17
  4. 2021Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descentcited by 16
  5. 2021A Continuized View on Nesterov Acceleration for Stochastic Gradient Descent and Randomized Gossipcited by 15
  6. 2021Robust Regression Revisited: Acceleration and Improved Estimation Ratescited by 15
  7. 2021Semi-Discrete Optimal Transport: Hardness, Regularization and Numerical Solutioncited by 13
  8. 2021Complementary Composite Minimization, Small Gradients in General Norms, and Applicationscited by 9
  9. 2021Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimizationcited by 8
  10. 2021Random-reshuffled SARAH does not need a full gradient computationscited by 7
  11. 2021Parameter-free Locally Accelerated Conditional Gradientscited by 5
  12. 2021First-Order Methods for Convex Optimizationcited by 4
8 more
  1. 2023Accelerated Cyclic Coordinate Dual Averaging with Extrapolation for Composite Convex Optimizationcited by 3
  2. 2023On the Identification and Optimization of Nonsmooth Superposition Operators in Semilinear Elliptic PDEscited by 2
  3. 2024A Mirror Descent-Based Algorithm for Corruption-Tolerant Distributed Gradient Descentcited by 2
  4. 2022Tradeoffs between convergence rate and noise amplification for momentum-based accelerated optimization algorithmscited by 1
  5. 2022BSAC-CoEx: Coexistence of URLLC and Distributed Learning Services via Device Selectioncited by 1
  6. 2024Optimization on a Finer Scale: Bounded Local Subgradient Variation Perspectivecited by 1
  7. 2021A High-order Tuner for Accelerated Learning and Controlcited by 0
  8. 2024Faster Convergence of Stochastic Accelerated Gradient Descent under Interpolationcited by 0