mlsanityData to 7 Oct 2026

94% on CIFAR-10 in 3.29 Seconds on a Single GPU

arXiv 2404.00498 ↗First posted 30 Mar 2024cs.LG cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

CIFAR-10 is among the most widely used datasets in machine learning, facilitating thousands of research projects per year. To accelerate research and reduce the cost of experiments, we introduce training methods for CIFAR-10 which reach 94% accuracy in 3.29 seconds, 95% in 10.4 seconds, and 96% in 46.3 seconds, when run on a single NVIDIA A100 GPU. As one factor contributing to these training speeds, we propose a derandomized variant of horizontal flipping augmentation, which we show improves over the standard method in every case where flipping is beneficial over no flipping at all. Our code is released at https://github.com/KellerJordan/cifar10-airbench.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 14 of its 25 references that are in the map. First posted on arXiv 30 Mar 2024.

Builds on

14 of 25 references.

References · 14

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2014Going Deeper with Convolutionscited by 4,231
  4. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  5. 2016Wide Residual Networkscited by 3,057
  6. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  7. 2019Lookahead Optimizer: k steps forward, 1 step backcited by 333
  8. 2019Fixing the train-test resolution discrepancycited by 209
  9. 2022Datamodels: Predicting Predictions from Training Datacited by 186
  10. 2021Assessing Generalization of SGD via Disagreementcited by 116
  11. 2015Why Random Reshuffling Beats Stochastic Gradient Descentcited by 78
  12. 2023FFCV: Accelerating Training by Removing Data Bottleneckscited by 49
2 more
  1. 2022Benchopt: Reproducible, efficient and collaborative optimization benchmarkscited by 27
  2. 2023On the Variance of Neural Network Training with respect to Test Sets and Distributionscited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 10.

15 papers.

Sort:

By area:

  1. 2025Bayesian Influence Functions for Hessian-Free Data Attributioncited by 12
  2. 2026Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decompositioncited by 8
  3. 2026Beyond Muon: MUD (MomentUm Decorrelation) for Faster Transformer Trainingcited by 4
  4. 2026An Embarrassingly Simple Way to Optimize Orthogonal Matrices at Scalecited by 3
  5. 2025ENSAM: an efficient foundation model for interactive segmentation of 3D medical imagescited by 2
  6. 2026Scale-Invariant Neural Network Optimization: Norm Geometry and Heavy-Tailed Noisecited by 2
  7. 2026AgentHPOBench: A Benchmark For Evaluating LLM Agents as Sequential Hyperparameter Optimizerscited by 2
  8. 2025Powerful Design of Small Vision Transformer on CIFAR10cited by 1
  9. 2025Steganographic Embeddings as an Effective Data Augmentationcited by 0
  10. 2025FRAIN to Train: A Fast-and-Reliable Solution for Decentralized Federated Learningcited by 0
  11. 2026Ctrl-A: Control-Driven Online Data Augmentationcited by 0
  12. 2026When to use what Schatten-p norm in deep learning?cited by 0
3 more
  1. 2026Nonsmooth Optimization via Orthogonalized Momentumcited by 0
  2. 2026The Row Normalization Puzzle in Muoncited by 0
  3. 2026Spectrally Targeted Muoncited by 0