mlsanityData to 14 Oct 2026

PolyThrottle: Energy-efficient Neural Network Inference on Edge Devices

arXiv 2310.19991 ↗First posted 30 Oct 2023cs.LG cs.ARcited by 5

Not on the map. Why?

Abstract (arXiv)

As neural networks (NN) are deployed across diverse sectors, their energy demand correspondingly grows. While several prior works have focused on reducing energy consumption during training, the continuous operation of ML-powered systems leads to significant energy use during inference. This paper investigates how the configuration of on-device hardware-elements such as GPU, memory, and CPU frequency, often neglected in prior studies, affects energy consumption for NN inference with regular fine-tuning. We propose PolyThrottle, a solution that optimizes configurations across individual hardware components using Constrained Bayesian Optimization in an energy-conserving manner. Our empirical evaluation uncovers novel facets of the energy-performance equilibrium showing that we can save up to 36 percent of energy for popular models. We also validate that PolyThrottle can quickly converge towards near-optimal settings while satisfying application constraints.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 36 of its 69 references that are in the map. First posted on arXiv 30 Oct 2023.

Builds on

36 of 69 references.

References · 36

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  3. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  6. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  7. 2019Searching for MobileNetV3cited by 1,994
  8. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  9. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  10. 2019Green AIcited by 627
  11. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  12. 2019Quantifying the Carbon Emissions of Machine Learningcited by 521
24 more
  1. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 300
  2. 2021I-BERT: Integer-only BERT Quantizationcited by 255
  3. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  4. 2016Clipper: A Low-Latency Online Prediction Serving Systemcited by 191
  5. 2020Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Modelscited by 169
  6. 2018Scalable Methods for 8-bit Training of Neural Networkscited by 142
  7. 2020Serving DNNs like Clockwork: Performance Predictability from the Bottom Upcited by 120
  8. 2020Data Movement Is All You Need: A Case Study on Optimizing Transformerscited by 108
  9. 2022Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Trainingcited by 79
  10. 2020Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learningcited by 72
  11. 2020Chasing Carbon: The Elusive Environmental Footprint of Computingcited by 72
  12. 2020EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inferencecited by 70
  13. 2019Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonflycited by 67
  14. 2018A First Look at Deep Learning Apps on Smartphonescited by 47
  15. 2019Blink: Fast and Generic Collectives for Distributed MLcited by 36
  16. 2019On-Device Neural Net Inference with Mobile GPUscited by 32
  17. 2020Towards Accurate and Reliable Energy Measurement of NLP Modelscited by 16
  18. 2020The gem5 Simulator: Version 20.0+cited by 14
  19. 2022Dynamic GPU Energy Optimization for Machine Learning Training Workloadscited by 12
  20. 2022Treehouse: A Case For Carbon-Aware Datacenter Softwarecited by 12
  21. 2020Verified Instruction-Level Energy Consumption Measurement for NVIDIA GPUscited by 10
  22. 2023MuxFlow: Efficient and Safe GPU Sharing in Large-Scale Production Deep Learning Clusterscited by 7
  23. 2016A Survey and Measurement Study of GPU DVFS on Energy Conservationcited by 6
  24. 2023Energy-Efficient GPU Clusters Scheduling for Deep Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Covariance-Guided Resource Adaptive Learning for Efficient Edge Inferencecited by 0
  2. 2026A Comparative Study of CNN Optimization Methods for Edge AI: Exploring the Role of Early Exitscited by 0
  3. 2026Strait: Perceiving Priority and Interference in ML Inference Servingcited by 0
  4. 2026Formally Verifying Analog Neural Networks Under Process Variations Using Polynomial Zonotopescited by 0
  5. 2026Beyond Independent Optimization: Compression, MoE Routing, and Quantization Interactions in Multimodal Edge Intelligencecited by 0