# Power Consumption Variation over Activation Functions

arXiv: https://arxiv.org/abs/2006.07237 · HTML: https://mlsanity.com/paper/2006.07237/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2020.

## Built on by (most cited first, 4 of 4)

- [Efficient Methods for Natural Language Processing: A Survey](https://mlsanity.com/paper/2209.00099/) (2022)
- [On the Limitations of Compute Thresholds as a Governance Strategy](https://mlsanity.com/paper/2407.05694/) (2024)
- [DeepHGCN: Toward Deeper Hyperbolic Graph Convolutional Networks](https://mlsanity.com/paper/2310.02027/) (2023)
- [The HCI GenAI CO2ST Calculator: A Tool for Calculating the Carbon Footprint of Generative AI Use in Human-Computer Interaction Research](https://mlsanity.com/paper/2504.00692/) (2025)

## Abstract (arXiv)

The power that machine learning models consume when making predictions can be affected by a model's architecture. This paper presents various estimates of power consumption for a range of different activation functions, a core factor in neural network model architecture design. Substantial differences in hardware performance exist between activation functions. This difference informs how power consumption in machine learning models can be reduced.