# Algorithmic progress in computer vision

arXiv: https://arxiv.org/abs/2212.05153 · HTML: https://mlsanity.com/paper/2212.05153/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 20 references that are in the map.
- First posted on arXiv 10 Dec 2022.

## Builds on (most cited first, 6 of 6)

- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [The Computational Limits of Deep Learning](https://mlsanity.com/paper/2007.05558/) (2020)
- [All Tokens Matter: Token Labeling for Training Better Vision Transformers](https://mlsanity.com/paper/2104.10858/) (2021)
- [Measuring the Algorithmic Efficiency of Neural Networks](https://mlsanity.com/paper/2005.04305/) (2020)
- [Progress in Mathematical Programming Solvers from 2001 to 2020](https://mlsanity.com/paper/2206.09787/) (2022)
- [A Time Leap Challenge for SAT Solving](https://mlsanity.com/paper/2008.02215/) (2020)

## Built on by (most cited first, 14 of 14)

- [How predictable is language model benchmark performance?](https://mlsanity.com/paper/2401.04757/) (2024)
- [A Rosetta Stone for AI Benchmarks](https://mlsanity.com/paper/2512.00193/) (2025)
- [Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives](https://mlsanity.com/paper/2311.09227/) (2023)
- [Algorithmic progress in language models](https://mlsanity.com/paper/2403.05812/) (2024)
- [What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring](https://mlsanity.com/paper/2303.11341/) (2023)
- [Training Compute Thresholds: Features and Functions in AI Regulation](https://mlsanity.com/paper/2405.10799/) (2024)
- [International Governance of Civilian AI: A Jurisdictional Certification Approach](https://mlsanity.com/paper/2308.15514/) (2023)
- [On the Origin of Algorithmic Progress in AI](https://mlsanity.com/paper/2511.21622/) (2025)
- [Trends in AI Supercomputers](https://mlsanity.com/paper/2504.16026/) (2025)
- [From AGI to ASI](https://mlsanity.com/paper/2606.12683/) (2026)
- [The Graph's Apprentice: Teaching an LLM Low Level Knowledge for Circuit Quality Estimation](https://mlsanity.com/paper/2411.00843/) (2024)
- [Rethinking LLM Advancement: Compute-Dependent and Independent Paths to Progress](https://mlsanity.com/paper/2505.04075/) (2025)
- [Beyond Tokens: Enhancing RTL Quality Estimation via Structural Graph Learning](https://mlsanity.com/paper/2508.18730/) (2025)
- [Will Neural Scaling Laws Activate Jevons' Paradox in AI Labor Markets? A Time-Varying Elasticity of Substitution (VES) Analysis](https://mlsanity.com/paper/2503.05816/) (2025)

## Abstract (arXiv)

We investigate algorithmic progress in image classification on ImageNet, perhaps the most well-known test bed for computer vision. We estimate a model, informed by work on neural scaling laws, and infer a decomposition of progress into the scaling of compute, data, and algorithms. Using Shapley values to attribute performance improvements, we find that algorithmic improvements have been roughly as important as the scaling of compute for progress computer vision. Our estimates indicate that algorithmic innovations mostly take the form of compute-augmenting algorithmic advances (which enable researchers to get better performance from less compute), not data-augmenting algorithmic advances. We find that compute-augmenting algorithmic advances are made at a pace more than twice as fast as the rate usually associated with Moore's law. In particular, we estimate that compute-augmenting innovations halve compute requirements every nine months (95% confidence interval: 4 to 25 months).