mlsanityData to 7 Oct 2026

Input-Dependent Power Usage in GPUs

arXiv 2409.18324 ↗First posted 26 Sep 2024cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

GPUs are known to be power-hungry, and due to the boom in artificial intelligence, they are currently the major contributors to the high power demands of upcoming datacenters. Most GPU usage in these popular workloads consist of large general matrix-matrix multiplications (GEMMs), which have therefore been optimized to achieve high utilization of hardware resources. In this work, we show that modifying the input data to GEMMs, while maintaining the matrix shapes and sizes can notably change the power consumption of these kernels. We experiment with four kinds of input variations: value distribution, bit similarity, placement, and sparsity, across different data types. Our findings indicate that these variations can change the GPU power usage during GEMM by almost 40%. We hypothesize that input-dependent power usage variations occur due to changes in the number of bit flips in the GPUs. We propose leveraging this property through compiler and scheduler optimizations to manage power and reduce energy consumption.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 15 of its 45 references that are in the map. First posted on arXiv 26 Sep 2024.

Builds on

15 of 45 references.

References · 15

  1. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  2. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  3. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
  4. 2017Efficient Processing of Deep Neural Networks: A Tutorial and Surveycited by 427
  5. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
  6. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  7. 2020Sparse GPU Kernels for Deep Learningcited by 138
  8. 2022Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Trainingcited by 78
  9. 2023TorchSparse++: Efficient Training and Inference Framework for Sparse Convolution on GPUscited by 36
  10. 2023PIT: Optimization of Dynamic Sparse Deep Learning Models via Permutation Invariant Transformationcited by 25
  11. 2023Reducing Energy Bloat in Large Model Trainingcited by 22
  12. 2023Exploding AI Power Use: an Opportunity to Rethink Grid Planning and Managementcited by 7
3 more
  1. 2023Adapting Datacenter Capacity for Greener Datacenters and Gridcited by 6
  2. 2022Understanding the Impact of Input Entropy on FPU, CPU, and GPU Powercited by 1
  3. 2023A Survey of FPGA Optimization Methods for Data Center Energy Efficiencycited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Privacy-Preserving AI Verification via Minimal Information Disclosurecited by 0