mlsanityData to 14 Oct 2026

Trends in AI Supercomputers

arXiv 2504.16026 ↗First posted 22 Apr 2025cs.CY cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Frontier AI development relies on powerful AI supercomputers, yet analysis of these systems is limited. We create a dataset of 500 AI supercomputers from 2019 to 2025 and analyze key trends in performance, power needs, hardware cost, ownership, and global distribution. We find that the computational performance of AI supercomputers has doubled every nine months, while hardware acquisition cost and power needs both doubled every year. The leading system in March 2025, xAI's Colossus, used 200,000 AI chips, had a hardware cost of 7B, and required 300 MW of power, as much as 250,000 households. As AI supercomputers evolved from tools for science to industrial machines, companies rapidly expanded their share of total AI supercomputer performance, while the share of governments and academia diminished. Globally, the United States accounts for about 75% of total performance in our dataset, with China in second place at 15%. If the observed trends continue, the leading AI supercomputer in 2030 will achieve 2×1022 16-bit FLOP/s, use two million AI chips, have a hardware cost of 200 billion, and require 9 GW of power. Our analysis provides visibility into the AI supercomputer landscape, allowing policymakers to assess key AI trends like resource needs, ownership, and national competitiveness.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 13 of its 109 references that are in the map. First posted on arXiv 22 Apr 2025.

Builds on

13 of 109 references.

References · 13

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2017Mixed Precision Trainingcited by 1,156
  3. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 632
  4. 2019MLPerf Training Benchmarkcited by 115
  5. 2024The rising costs of training frontier AI modelscited by 95
  6. 2024Computing Power and the Governance of Artificial Intelligencecited by 40
  7. 2023QUIK: Towards End-to-End 4-Bit Inference on Generative Large Language Modelscited by 39
  8. 2024The Compute Divide in Machine Learning: A Threat to Academic Contribution and Scrutiny?cited by 20
  9. 2022Algorithmic progress in computer visioncited by 14
  10. 2023Nuclear Arms Control Verification and Lessons for AI Treatiescited by 6
  11. 2022A Brief History of Recommender Systemscited by 4
  12. 2023Compute at Scale: A Broad Investigation into the Data Center Industrycited by 2
1 more
  1. 2024Green HPC: An analysis of the domain based on Top500cited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.