mlsanityData to 7 Oct 2026

Nesterov's Accelerated Gradient and Momentum as approximations to Regularised Update Descent

arXiv 1607.01981 ↗First posted 7 Jul 2016stat.ML cs.LGcited by 26

Not on the map. Why?

Abstract (arXiv)

We present a unifying framework for adapting the update direction in gradient-based iterative optimization methods. As natural special cases we re-derive classical momentum and Nesterov's accelerated gradient method, lending a new intuitive interpretation to the latter algorithm. We show that a new algorithm, which we term Regularised Gradient Descent, can converge more quickly than either Nesterov's algorithm or the classical momentum algorithm.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Jul 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 4; 2023: 7; 2024: 5; 2025: 4; 2026: 2.

26 papers.

Sort:

  1. 2023Multi-modal Learning with Missing Modality via Shared-Specific Feature Modellingcited by 159
  2. 2023Learnable Cross-modal Knowledge Distillation for Multi-modal Learning with Missing Modalitycited by 44
  3. 2022Learning from Future: A Novel Self-Training Framework for Semantic Segmentationcited by 21
  4. 2021Correcting Momentum in Temporal Difference Learningcited by 12
  5. 2023Towards a universal mechanism for successful deep learningcited by 8
  6. 2022HRel: Filter Pruning based on High Relevance between Activation Maps and Class Labelscited by 7
  7. 2022Efficient shallow learning as an alternative to deep learningcited by 6
  8. 2024Meta-Learned Modality-Weighted Knowledge Distillation for Robust Multi-Modal Learning with Missing Datacited by 5
  9. 2025DATA: Multi-Disentanglement based Contrastive Learning for Open-World Semi-Supervised Deepfake Attributioncited by 5
  10. 2023The mechanism underlying successful deep learningcited by 4
  11. 2025Clothes-Changing Person Re-identification Based On Skeleton Dynamicscited by 4
  12. 2023Enhancing the accuracies by performing pooling decisions adjacent to the output layercited by 3
14 more
  1. 2023Can we learn better with hard samples?cited by 3
  2. 2024Robust Semi-supervised Learning via f-Divergence and α-R'enyi Divergencecited by 3
  3. 2021Improving Adversarial Robustness for Free with Snapshot Ensemblecited by 1
  4. 2021UAV-based Crowd Surveillance in Post COVID-19 Eracited by 1
  5. 2023The Disharmony between BN and ReLU Causes Gradient Explosion, but is Offset by the Correlation between Activationscited by 1
  6. 2024Role of Delay in Brain Dynamicscited by 1
  7. 2024Error estimates between SGD with momentum and underdamped Langevin diffusioncited by 1
  8. 2021DNA: Dynamic Network Augmentationcited by 0
  9. 2022Two-stream Multi-dimensional Convolutional Network for Real-time Violence Detectioncited by 0
  10. 2024A Scientific Machine Learning Approach for Predicting and Forecasting Battery Degradation in Electric Vehiclescited by 0
  11. 2025Enhanced NIRMAL Optimizer With Damped Nesterov Acceleration: A Comparative Analysiscited by 0
  12. 2025A Proportional-Integral Controller-Incorporated SGD Algorithm for High Efficient Latent Factor Analysiscited by 0
  13. 2026Riemannian Lyapunov Optimizer: A Unified Framework for Optimizationcited by 0
  14. 2026Token-UNet: A New Case for Transformers Integration in Efficient and Interpretable 3D UNets for Brain Imaging Segmentationcited by 0