mlsanityData to 14 Oct 2026

Enabling Spatially Fine-Grained DVFS in Neural Processing Units for Energy-Efficient LLM Serving

arXiv 2607.16473 ↗First posted 17 Jul 2026cs.ARcited by 0

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

As neural processing units (NPUs) evolve rapidly to accommodate the ever-increasing compute demand of large language models (LLMs), their power consumption is becoming a limiting factor. Our study shows that using dynamic voltage and frequency scaling (DVFS) to exploit the service-level objective (SLO) slacks is a promising way to improve NPU energy efficiency for LLM services. And as tensor operators in LLMs exhibit diverse bottlenecks across NPU components, it is desirable to configure the frequency separately for each component to maximize their energy efficiency. In this paper, we develop eNPU that enables hardware and software support for spatially fine-grained, component-level DVFS on NPUs. eNPU refactors the NPU core pipeline to partition components into separate V/f domains. It introduces lightweight cross-domain communication mechanisms to mitigate synchronization overheads across components, and extends the NPU ISA for sub-μs DVFS control. eNPU uses a compiler-driven two-level greedy search to co-optimize instruction scheduling and per-component V/f selection under SLO constraints. We implement eNPU's pipeline design on an open-source NPU core to verify its functionality and evaluate the energy savings with a production-level NPU simulator with various LLMs using production traces. eNPU reduces energy consumption of LLM services by 25.8%--35.2% with 3.45% area overhead on a TPUv4 chip, while preserving strict SLO guarantees.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 18 of its 70 references that are in the map. First posted on arXiv 17 Jul 2026.

Builds on

18 of 70 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026DualScale: Energy-Efficient Disaggregated LLM Serving via Phase-Aware Placement and DVFS21 Feb
  2. 2024SLO-aware GPU Frequency Scaling for Energy Efficient LLM Inference Serving5 Aug
  3. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve4 Mar
  4. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving18 Jan

Other references · 14

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  4. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
  5. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 532
  6. 2023TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddingscited by 202
  7. 2025TAPAS: Thermal- and Power-Aware Scheduling for LLM Inference in Cloud Platformscited by 29
  8. 2025EcoServe: Designing Carbon-Aware AI Inference Systemscited by 19
  9. 2025Capacity-Aware Inference: Mitigating the Straggler Effect in Mixture of Expertscited by 15
  10. 2023POLCA: Power Oversubscription in LLM Cloud Providerscited by 9
  11. 2022Predict; Do not React for Enabling Efficient Fine Grain DVFS in GPUscited by 3
  12. 2025AI Greenferencing: Routing AI Inferencing to Green Modular Data Centers with Heroncited by 3
2 more
  1. 2025ReGate: Enabling Power Gating in Neural Processing Unitscited by 3
  2. 2026Analysis of Edge Mismatch and Output Power Degradation in Cascoded Class-D Power Amplifiers Using Dual-Range Voltage Level Shifterscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Auto-Scaling Heterogeneous Neural Processing Units for Energy and Cost-Efficient LLM Serving
  2. 2026Energy-Efficient LLM Serving via Disaggregated Attention--FFN and Flexible Frequency Scaling
  3. 2026When NPUs Are Not Always Faster: A Stage-Level Analysis of Mobile LLM Inference
  4. 2026PALS: Power-Aware LLM Serving for Mixture-of-Experts Models
  5. 2024Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference
  6. 2025VoltanaLLM: Energy-Efficient and SLO-Aware Disaggregated LLM Serving via Adaptive Frequency Control and State-Space Routing
  7. 2026Understanding Efficiency: Quantization, Batching, and Serving Strategies in LLM Energy Use
  8. 2025Characterizing LLM Inference Energy-Performance Tradeoffs across Workloads and GPU Scaling