mlsanityData to 7 Oct 2026

Offline Energy-Optimal LLM Serving: Workload-Based Energy Models for LLM Inference on Heterogeneous Systems

arXiv 2407.04014 ↗First posted 4 Jul 2024cs.DCcited by 27

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

The rapid adoption of large language models (LLMs) has led to significant advances in natural language processing and text generation. However, the energy consumed through LLM model inference remains a major challenge for sustainable AI deployment. To address this problem, we model the workload-dependent energy consumption and runtime of LLM inference tasks on heterogeneous GPU-CPU systems. By conducting an extensive characterization study of several state-of-the-art LLMs and analyzing their energy and runtime behavior across different magnitudes of input prompts and output text, we develop accurate (R^2>0.96) energy and runtime models for each LLM. We employ these models to explore an offline, energy-optimal LLM workload scheduling framework. Through a case study, we demonstrate the advantages of energy and accuracy aware scheduling compared to existing best practices.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 23 of its 51 references that are in the map. First posted on arXiv 4 Jul 2024.

Builds on

23 of 51 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline22 May

Other references · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2024Mixtral of Expertscited by 1,785
  4. 2021Carbon Emissions and Large Neural Network Trainingcited by 561
  5. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
  6. 2022Efficiently Scaling Transformer Inferencecited by 529
  7. 2022Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Modelcited by 264
  8. 2020Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learningcited by 246
  9. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  10. 2023From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inferencecited by 168
  11. 2021Compute and Energy Consumption Trends in Deep Learning Inferencecited by 61
  12. 2024Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inferencecited by 59
10 more
  1. 2021Characterization and Prediction of Deep Learning Workloads in Large-Scale GPU Datacenterscited by 31
  2. 2024Hybrid Heterogeneous Clusters Can Lower the Energy Consumption of LLM Inference Workloadscited by 17
  3. 2023Clover: Toward Sustainable AI with Carbon-Aware Machine Learning Inference Servicecited by 16
  4. 2024Toward Sustainable GenAI using Generation Directives for Carbon-Friendly Large Language Model Inferencecited by 13
  5. 2022Treehouse: A Case For Carbon-Aware Datacenter Softwarecited by 12
  6. 2023Towards Environmentally Equitable AI via Geographical Load Balancingcited by 10
  7. 2023Exploding AI Power Use: an Opportunity to Rethink Grid Planning and Managementcited by 7
  8. 2022Model-driven Cluster Resource Management for AI Workloads in Edge Cloudscited by 6
  9. 2023Adapting Datacenter Capacity for Greener Datacenters and Gridcited by 6
  10. 2023Energy-Efficient GPU Clusters Scheduling for Deep Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 7; 2026: 18.

27 papers.

Sort:

By area:

  1. 2024On-Device Language Models: A Comprehensive Reviewcited by 101
  2. 2025Energy Considerations of Large Language Model Inference and Efficiency Optimizationscited by 54
  3. 2024SLO-aware GPU Frequency Scaling for Energy Efficient LLM Inference Servingcited by 18
  4. 2025From Prompts to Power: Measuring the Energy Footprint of LLM Inferencecited by 12
  5. 2025Distributed LLMs and Multimodal Large Language Models: A Survey on Advances, Challenges, and Future Directionscited by 10
  6. 2026SweetSpot: An Analytical Model for Predicting Energy Efficiency of LLM Inferencecited by 6
  7. 2025SWEnergy: An Empirical Study on Energy Efficiency in Agentic Issue Resolution Frameworks with SLMscited by 5
  8. 2026Beyond Test-Time Compute Strategies: Advocating Energy-per-Token in LLM Inferencecited by 5
  9. 2026Understanding Efficiency: Quantization, Batching, and Serving Strategies in LLM Energy Usecited by 4
  10. 2025GPT-OSS-20B: A Comprehensive Deployment-Centric Analysis of OpenAI's Open-Weight Mixture of Experts Modelcited by 2
  11. 2025Revisiting Disaggregated Large Language Model Serving for Performance and Energy Implicationscited by 2
  12. 2026Characterizing Performance-Energy Trade-offs of Large Language Models in Multi-Request Workflowscited by 2
15 more
  1. 2026Characterization of Request and Token Energy Costs for LLM Inference Workloads on GPU Platformscited by 2
  2. 2025Addressing the sustainable AI trilemma: a case study on LLM agents and RAGcited by 1
  3. 2026Small Talk, Big Impact: The Energy Cost of Thanking AIcited by 1
  4. 2026Pimp My LLM: Leveraging Variability Modeling to Tune Inference Hyperparameterscited by 1
  5. 2026KAIROS: Stateful, Context-Aware Power-Efficient Agentic Inference Servingcited by 1
  6. 2026WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMscited by 1
  7. 2026Attention to Detail: Evaluating Energy, Performance, and Accuracy Trade-offs Across vLLM Configurationscited by 1
  8. 2026Token Management in Multi-Tenant AI Inference Platformscited by 0
  9. 2026SEALing the Gap: A Reference Framework for LLM Inference Carbon Estimation via Multi-Benchmark Driven Embodimentcited by 0
  10. 2026From Perception to Action: Can UI Interventions Foster Sustainable LLM Chatbotcited by 0
  11. 2026From Caveman to Expert Analyst: Energy Consumption of Variable LLM Taskscited by 0
  12. 2026TokenPowerSandbox: Evidence-Gated CPU-First Screening for Energy-Aware LLM Servingcited by 0
  13. 2026Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scalingcited by 0
  14. 2026Analytical Power-Aware Provisioning for Prefill-Decode Disaggregated AI Inferencecited by 0
  15. 2026Greenpixie's AI Token Methodology: Assessing the Energy, Water and CO2-eq Impact of AI Tokens for Open and Closed Weight Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Characterizing LLM Inference Energy-Performance Tradeoffs across Workloads and GPU Scaling
  2. 2026Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures
  3. 2024DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency
  4. 2025Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latency
  5. 2025Benchmarking Energy Efficiency of Large Language Models Using vLLM
  6. 2025The Energy Cost of Reasoning: Analyzing Energy Usage in LLMs with Test-time Compute
  7. 2024NEO: Saving GPU Memory Crisis with CPU Offloading for Online LLM Inference
  8. 2026PALS: Power-Aware LLM Serving for Mixture-of-Experts Models