mlsanityData to 7 Oct 2026

PALS: Power-Aware LLM Serving for Mixture-of-Experts Models

arXiv 2605.21427 ↗First posted 20 May 2026cs.AI cs.DCcited by 1

On the map: Mixture-of-experts → Efficient MoE training and inference systems

also in 1 more
Abstract (arXiv)

Large language model (LLM) inference has become a dominant workload in modern data centers, driving significant GPU utilization and energy consumption. While prior systems optimize throughput and latency by batching, scheduling, and parallelism, they largely treat GPU power as a static constraint rather than a controllable resource. In this paper, we present a power-aware runtime for LLM serving, PALS, that treats GPU power caps as a first-class control knob and jointly optimizes them with software parameters such as batch size. The system combines lightweight offline power-performance models with a feedback-driven controller to select configurations that satisfy throughput targets while maximizing energy efficiency. We implement PALS within an existing LLM serving framework, vLLM, demonstrating that it requires no model retraining or API changes. Across multi-GPU systems and both dense and mixture-of-experts (MoE) models, PALS improves energy efficiency by up to 26.3%, reduces QoS violations by 4x to 7x under power constraints, and tracks dynamic power budgets. These results highlight the potential of integrating power control directly into LLM inference runtimes, enabling energy-proportional and grid-interactive AI systems.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 18 of its 34 references that are in the map. First posted on arXiv 20 May 2026.

Builds on

18 of 34 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Fast Inference of Mixture-of-Experts Language Models with Offloading28 Dec

Other references · 17

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  6. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  7. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  8. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  9. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  10. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 621
  11. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
  12. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 507
5 more
  1. 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 403
  2. 2022Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Trainingcited by 78
  3. 2024Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flowcited by 54
  4. 2024Don't Stop Me Now: Embedding Based Scheduling for LLMscited by 34
  5. 2022Carbon Explorer: A Holistic Approach for Designing Carbon Aware Datacenterscited by 31

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Phase-Decoupled, Model-Calibrated Power Control for Disaggregated LLM Servingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures
  2. 2024Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference
  3. 2025Characterizing LLM Inference Energy-Performance Tradeoffs across Workloads and GPU Scaling
  4. 2026EnergyLens: Predictive Energy-Aware Exploration for Multi-GPU LLM Inference Optimization
  5. 2024Offline Energy-Optimal LLM Serving: Workload-Based Energy Models for LLM Inference on Heterogeneous Systems
  6. 2024SLO-aware GPU Frequency Scaling for Energy Efficient LLM Inference Serving
  7. 2025MoE-Lens: Towards the Hardware Limit of High-Throughput MoE LLM Serving Under Resource Constraints
  8. 2025Parallax: Efficient LLM Inference Service over Decentralized Environment