# Input-Dependent Power Usage in GPUs

arXiv: https://arxiv.org/abs/2409.18324 · HTML: https://mlsanity.com/paper/2409.18324/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 45 references that are in the map.
- First posted on arXiv 26 Sep 2024.

## Builds on (most cited first, 15 of 15)

- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Splitwise: Efficient generative LLM inference using phase splitting](https://mlsanity.com/paper/2311.18677/) (2023)
- [Efficient Processing of Deep Neural Networks: A Tutorial and Survey](https://mlsanity.com/paper/1703.09039/) (2017)
- [SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression](https://mlsanity.com/paper/2306.03078/) (2023)
- [Sustainable AI: Environmental Implications, Challenges and Opportunities](https://mlsanity.com/paper/2111.00364/) (2021)
- [Sparse GPU Kernels for Deep Learning](https://mlsanity.com/paper/2006.10901/) (2020)
- [Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Training](https://mlsanity.com/paper/2208.06102/) (2022)
- [TorchSparse++: Efficient Training and Inference Framework for Sparse Convolution on GPUs](https://mlsanity.com/paper/2311.12862/) (2023)
- [PIT: Optimization of Dynamic Sparse Deep Learning Models via Permutation Invariant Transformation](https://mlsanity.com/paper/2301.10936/) (2023)
- [Reducing Energy Bloat in Large Model Training](https://mlsanity.com/paper/2312.06902/) (2023)
- [Exploding AI Power Use: an Opportunity to Rethink Grid Planning and Management](https://mlsanity.com/paper/2311.11645/) (2023)
- [Adapting Datacenter Capacity for Greener Datacenters and Grid](https://mlsanity.com/paper/2301.03148/) (2023)
- [Understanding the Impact of Input Entropy on FPU, CPU, and GPU Power](https://mlsanity.com/paper/2212.08805/) (2022)
- [A Survey of FPGA Optimization Methods for Data Center Energy Efficiency](https://mlsanity.com/paper/2309.12884/) (2023)

## Built on by (most cited first, 1 of 1)

- [Privacy-Preserving AI Verification via Minimal Information Disclosure](https://mlsanity.com/paper/2608.02774/) (2026)

## Abstract (arXiv)

GPUs are known to be power-hungry, and due to the boom in artificial intelligence, they are currently the major contributors to the high power demands of upcoming datacenters. Most GPU usage in these popular workloads consist of large general matrix-matrix multiplications (GEMMs), which have therefore been optimized to achieve high utilization of hardware resources. In this work, we show that modifying the input data to GEMMs, while maintaining the matrix shapes and sizes can notably change the power consumption of these kernels. We experiment with four kinds of input variations: value distribution, bit similarity, placement, and sparsity, across different data types. Our findings indicate that these variations can change the GPU power usage during GEMM by almost 40%. We hypothesize that input-dependent power usage variations occur due to changes in the number of bit flips in the GPUs. We propose leveraging this property through compiler and scheduler optimizations to manage power and reduce energy consumption.