mlsanityData to 14 Oct 2026

MLPerf Training Benchmark

arXiv 1910.01500 ↗First posted 2 Oct 2019cs.LG cs.PF stat.MLcited by 115

Not on the map. Why?

Abstract (arXiv)

Machine learning (ML) needs industry-standard performance benchmarks to support design and competitive evaluation of the many emerging software and hardware solutions for ML. But ML training presents three unique benchmarking challenges absent from other domains: optimizations that improve training throughput can increase the time to solution, training is stochastic and time to solution exhibits high variance, and software and hardware systems are so diverse that fair benchmarking with the same binary, code, and even hyperparameters is difficult. We therefore present MLPerf, an ML benchmark that overcomes these challenges. Our analysis quantitatively evaluates MLPerf's efficacy at driving performance and scalability improvements across two rounds of results from multiple vendors.

Key facts. Cited by 115 papers in the map (data through 14 Oct 2026). First posted on arXiv 2 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 33; 2022: 21; 2023: 13; 2024: 14; 2025: 16; 2026: 18.

115 papers.

Sort:

By area:

  1. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  2. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 632
  3. 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 315
  4. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  5. 2021FedScale: Benchmarking Model and System Performance of Federated Learning at Scalecited by 136
  6. 2022Towards Efficient and Scalable Sharpness-Aware Minimizationcited by 129
  7. 2022Monarch: Expressive Structured Matrices for Efficient and Accurate Trainingcited by 106
  8. 2021The Efficiency Misnomercited by 98
  9. 2022DataPerf: Benchmarks for Data-Centric AI Developmentcited by 90
  10. 2021MedPerf: Open Benchmarking Platform for Medical Artificial Intelligence using Federated Evaluationcited by 88
  11. 2021Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Modelscited by 78
  12. 2024Introducing v0.5 of the AI Safety Benchmark from MLCommonscited by 62
38 more
  1. 2025Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluationcited by 60
  2. 2021Randomness In Neural Network Training: Characterizing The Impact of Toolingcited by 59
  3. 2021TT-Rec: Tensor Train Compression for Deep Learning Recommendation Modelscited by 58
  4. 2022VeLO: Training Versatile Learned Optimizers by Scaling Upcited by 57
  5. 2023FFCV: Accelerating Training by Removing Data Bottleneckscited by 49
  6. 2021tf.data: A Machine Learning Data Processing Frameworkcited by 41
  7. 2021AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learningcited by 39
  8. 2022MoESys: A Distributed and Efficient Mixture-of-Experts Training and Inference System for Internet Servicescited by 37
  9. 2021Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devicescited by 36
  10. 2022Distributed Graph Neural Network Training: A Surveycited by 32
  11. 2021A Runtime-Based Computational Performance Predictor for Deep Neural Network Trainingcited by 29
  12. 2021Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Trainingcited by 28
  13. 2022RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendationcited by 28
  14. 2022Aryl: An Elastic Cluster Scheduler for Deep Learningcited by 27
  15. 2023NeuroBench: A Framework for Benchmarking Neuromorphic Computing Algorithms and Systemscited by 27
  16. 2023A Machine Learning-oriented Survey on Tiny Machine Learningcited by 27
  17. 2021On the Utility of Gradient Compression in Distributed Training Systemscited by 26
  18. 2021A Generalizable Approach to Learning Optimizerscited by 23
  19. 2025TokenPowerBench: Benchmarking the Power Consumption of LLM Inferencecited by 22
  20. 2023A Survey on Deep Learning Hardware Accelerators for Heterogeneous HPC Platformscited by 21
  21. 2021A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizescited by 20
  22. 2021Plumber: Diagnosing and Removing Performance Bottlenecks in Machine Learning Data Pipelinescited by 16
  23. 2023TpuGraphs: A Performance Prediction Dataset on Large Tensor Computational Graphscited by 16
  24. 2024MLPerf Power: Benchmarking the Energy Efficiency of Machine Learning Systems from Microwatts to Megawatts for Sustainable AIcited by 16
  25. 2021User Response Prediction in Online Advertisingcited by 15
  26. 2021MLPerf HPC: A Holistic Benchmark Suite for Scientific Machine Learning on HPC Systemscited by 15
  27. 2021OneFlow: Redesign the Distributed Deep Learning Framework from Scratchcited by 15
  28. 2021Improving Computational Efficiency in Visual Reinforcement Learning via Stored Embeddingscited by 12
  29. 2022Benchmarking of DL Libraries and Models on Mobile Devicescited by 12
  30. 2021Concurrent Adversarial Learning for Large-Batch Trainingcited by 11
  31. 2021Prediction Surface Uncertainty Quantification in Object Detection Models for Autonomous Drivingcited by 10
  32. 2022Building a Performance Model for Deep Learning Recommendation Model Training on GPUscited by 10
  33. 2022BagPipe: Accelerating Deep Recommendation Model Trainingcited by 10
  34. 2022On Sampling Collaborative Filtering Datasetscited by 8
  35. 2023MoCA: Memory-Centric, Adaptive Execution for Multi-Tenant Deep Neural Networkscited by 8
  36. 2024Partial Rankings of Optimizerscited by 8
  37. 2025Greening AI-enabled Systems with Software Engineering: A Research Agenda for Environmentally Sustainable AI Practicescited by 8
  38. 2021Horizontally Fused Training Array: An Effective Hardware Utilization Squeezer for Training Novel Deep Learning Modelscited by 7

50 most cited of 115 All 115 (JSON)