mlsanityData to 13 Oct 2026

MLPerf Power: Benchmarking the Energy Efficiency of Machine Learning Systems from Microwatts to Megawatts for Sustainable AI

arXiv 2410.12032 ↗First posted 15 Oct 2024cs.AR cs.DC cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Rapid adoption of machine learning (ML) technologies has led to a surge in power consumption across diverse systems, from tiny IoT devices to massive datacenter clusters. Benchmarking the energy efficiency of these systems is crucial for optimization, but presents novel challenges due to the variety of hardware platforms, workload characteristics, and system-level interactions. This paper introduces MLPerf Power, a comprehensive benchmarking methodology with capabilities to evaluate the energy efficiency of ML systems at power levels ranging from microwatts to megawatts. Developed by a consortium of industry professionals from more than 20 organizations, MLPerf Power establishes rules and best practices to ensure comparability across diverse architectures. We use representative workloads from the MLPerf benchmark suite to collect 1,841 reproducible measurements from 60 systems across the entire range of ML deployment scales. Our analysis reveals trade-offs between performance, complexity, and energy efficiency across this wide range of systems, providing actionable insights for designing optimized ML solutions from the smallest edge devices to the largest cloud infrastructures. This work emphasizes the importance of energy efficiency as a key metric in the evaluation and comparison of the ML system, laying the foundation for future research in this critical area. We discuss the implications for developing sustainable AI solutions and standardizing energy efficiency benchmarking for ML systems.

Key facts. Cited by 15 papers in the map (data through 13 Oct 2026). Builds on 46 of its 91 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

46 of 91 references.

References · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  3. 2020Language Models are Few-Shot Learnerscited by 32,320
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,777
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,979
  6. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,836
  7. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,475
  8. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 617
  9. 2021Carbon Emissions and Large Neural Network Trainingcited by 565
  10. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  11. 2021A White Paper on Neural Network Quantizationcited by 467
  12. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 422
34 more
  1. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  2. 2023Multimodal Deep Learningcited by 257
  3. 2020Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learningcited by 249
  4. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  5. 2023TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddingscited by 202
  6. 2019MLPerf Inference Benchmarkcited by 197
  7. 2020Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Modelscited by 169
  8. 2023From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inferencecited by 168
  9. 2022Compute Trends Across Three Eras of Machine Learningcited by 163
  10. 2022On The Computational Complexity of Self-Attentioncited by 138
  11. 2021MLPerf Tiny Benchmarkcited by 122
  12. 2019MLPerf Training Benchmarkcited by 114
  13. 2017Hello Edge: Keyword Spotting on Microcontrollerscited by 108
  14. 2022Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Trainingcited by 78
  15. 2020Deep Audio-Visual Learning: A Surveycited by 68
  16. 2021Compute and Energy Consumption Trends in Deep Learning Inferencecited by 61
  17. 2024Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inferencecited by 61
  18. 2019MIMII Dataset: Sound Dataset for Malfunctioning Industrial Machine Investigation and Inspectioncited by 57
  19. 2018MEAL: Multi-Model Ensemble via Adversarial Learningcited by 52
  20. 2019ToyADMOS: A Dataset of Miniature-Machine Operating Sounds for Anomalous Sound Detectioncited by 44
  21. 2020Description and Discussion on DCASE2020 Challenge Task2: Unsupervised Anomalous Sound Detection for Machine Condition Monitoringcited by 43
  22. 2019PULP-NN: Accelerating Quantized Neural Networks on Parallel Ultra-Low-Power RISC-V Processorscited by 35
  23. 2022Carbon Explorer: A Holistic Approach for Designing Carbon Aware Datacenterscited by 31
  24. 2016Understanding the Impact of Precision Quantization on the Accuracy and Energy of Neural Networkscited by 18
  25. 2023Clover: Toward Sustainable AI with Carbon-Aware Machine Learning Inference Servicecited by 16
  26. 2021MLPerf HPC: A Holistic Benchmark Suite for Scientific Machine Learning on HPC Systemscited by 15
  27. 2020MLPerf Mobile Inference Benchmarkcited by 14
  28. 2023MAD Max Beyond Single-Node: Enabling Large Machine Learning Model Acceleration on Distributed Systemscited by 14
  29. 2021Reconsidering CO2 emissions from Computer Visioncited by 9
  30. 2019One Size Does Not Fit All: Quantifying and Exposing the Accuracy-Latency Trade-off in Machine Learning Cloud Service APIs via Tolerance Tierscited by 7
  31. 2023Carbon Responder: Coordinating Demand Response for the Datacenter Fleetcited by 5
  32. 2024Trillion Parameter AI Serving Infrastructure for Scientific Discovery: A Survey and Visioncited by 5
  33. 2023Is TinyML Sustainable? Assessing the Environmental Impacts of Machine Learning on Microcontrollerscited by 4
  34. 2024Enabling more efficient and cost-effective AI/ML systems with Collective Mind, virtualized MLOps, MLPerf, Collective Knowledge Playground and reproducible optimization tournamentscited by 3

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 7; 2026: 8.

15 papers.

Sort:

By area:

  1. 2025The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimizationcited by 31
  2. 2025TokenPowerBench: Benchmarking the Power Consumption of LLM Inferencecited by 22
  3. 2025Intelligence per Watt: Measuring Intelligence Efficiency of Local AIcited by 14
  4. 2026Where Do the Joules Go? Diagnosing Inference Energy Consumptioncited by 10
  5. 2026Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architecturescited by 5
  6. 2025AI Benchmark Democratization and Carpentrycited by 3
  7. 2025ELANA: A Simple Energy and Latency Analyzer for LLMscited by 2
  8. 2026Energy per Successful Goal: Goal-Level Energy Accounting for Agentic AI Systemscited by 1
  9. 2026WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMscited by 1
  10. 2025Lifetime-Aware Design for Item-Level Intelligence at the Extreme Edgecited by 0
  11. 2025Metrics and evaluations for computational and sustainable AI efficiencycited by 0
  12. 2026PrivacyBench: Privacy Isn't Free in Hybrid Privacy-Preserving Vision Systemscited by 0
3 more
  1. 2026OpenG2G: A Simulation Platform for AI Datacenter-Grid Runtime Coordinationcited by 0
  2. 2026Evaluation of ML Resource Utilization Requires Model Life Cycle Assessmentcited by 0
  3. 2026A Measurement Study of LLM Inference Trade-offs Across Edge Continuum Hardwarecited by 0