# Yet Another Accelerated SGD: ResNet-50 Training on ImageNet in 74.7 seconds

arXiv: https://arxiv.org/abs/1903.12650 · HTML: https://mlsanity.com/paper/1903.12650/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2019.

## Built on by (most cited first, 12 of 12)

- [Towards Efficient and Scalable Sharpness-Aware Minimization](https://mlsanity.com/paper/2203.02714/) (2022)
- [Stochastic Training is Not Necessary for Generalization](https://mlsanity.com/paper/2109.14119/) (2021)
- [Tesseract: Parallelize the Tensor Parallelism Efficiently](https://mlsanity.com/paper/2105.14500/) (2021)
- [Concurrent Adversarial Learning for Large-Batch Training](https://mlsanity.com/paper/2106.00221/) (2021)
- [Large-Scale Deep Learning Optimizations: A Comprehensive Survey](https://mlsanity.com/paper/2111.00856/) (2021)
- [Revisiting LARS for Large Batch Training Generalization of Neural Networks](https://mlsanity.com/paper/2309.14053/) (2023)
- [Accelerating Neural Network Training with Distributed Asynchronous and Selective Optimization (DASO)](https://mlsanity.com/paper/2104.05588/) (2021)
- [Large-batch Optimization for Dense Visual Predictions](https://mlsanity.com/paper/2210.11078/) (2022)
- [An Oracle for Guiding Large-Scale Model/Hybrid Parallel Training of Convolutional Neural Networks](https://mlsanity.com/paper/2104.09075/) (2021)
- [Energy Consumption in Parallel Neural Network Training](https://mlsanity.com/paper/2508.07706/) (2025)
- [Leveraging Computer Vision Application in Visual Arts: A Case Study on the Use of Residual Neural Network to Classify and Analyze Baroque Paintings](https://mlsanity.com/paper/2210.15300/) (2022)
- [Phasor-Driven Acceleration for FFT-based CNNs](https://mlsanity.com/paper/2406.00290/) (2024)

## Abstract (arXiv)

There has been a strong demand for algorithms that can execute machine learning as faster as possible and the speed of deep learning has accelerated by 30 times only in the past two years. Distributed deep learning using the large mini-batch is a key technology to address the demand and is a great challenge as it is difficult to achieve high scalability on large clusters without compromising accuracy. In this paper, we introduce optimization methods which we applied to this challenge. We achieved the training time of 74.7 seconds using 2,048 GPUs on ABCI cluster applying these methods. The training throughput is over 1.73 million images/sec and the top-1 validation accuracy is 75.08%.