mlsanityData to 14 Oct 2026

Request-Level Energy Attribution for Batched LLM Serving

arXiv 2608.00026 ↗First posted 11 Jul 2026cs.AI cs.DCcited by 1

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

Batched LLM serving improves throughput but complicates energy accounting. GPU power telemetry is aggregate, whereas sustainability reporting, chargeback, and workload analysis often require request-level energy charges. Existing inference-energy benchmarks report model-, phase-, or token-level energy, and recent carbon-accounting work motivates Shapley fairness conceptually. Neither provides measured request-level ground truth, so how far the accounting rules used in practice deviate from a fair allocation has remained unknown. We present JouleShare, an attribution framework with two components. An offline harness establishes this ground truth by replaying request subsets under vLLM with a reproducible protocol, integrating GPU power telemetry, and computing exact Shapley energy for each request. A lightweight calibration model, JCalib, then learns to predict Shapley shares from cheap request features for use at serving time. Across 16 model/workload runs, token-proportional attribution differs from exact Shapley by 0.440 normalized L1 on average under static batching and by 0.458 under continuous batching, a gap that reproduces across three data-center GPUs. JCalib reduces this error to 0.116 under static batching and 0.177 under continuous batching, below even a standalone-measurement baseline that is unavailable online, while preserving exact batch-energy efficiency. Sampled Shapley extends the measured reference to larger group sizes, where the gap persists and a single offline calibration remains the most accurate deployable rule. The results show that token attribution is not a reliable proxy for marginal energy under batched execution, and that measured Shapley ground truth can calibrate low-cost request features toward fairer attribution.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 16 of its 32 references that are in the map. First posted on arXiv 11 Jul 2026.

Builds on

16 of 32 references.

References · 16

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  5. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  6. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  7. 2019Green AIcited by 627
  8. 2021Carbon Emissions and Large Neural Network Trainingcited by 567
  9. 2020Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learningcited by 249
  10. 2022Measuring the Carbon Intensity of AI in Cloud Instancescited by 111
  11. 2025The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimizationcited by 33
  12. 2025TokenPowerBench: Benchmarking the Power Consumption of LLM Inferencecited by 22
4 more
  1. 2024LLMCO2: Advancing Accurate Carbon Footprint Prediction for LLM Inferencescited by 19
  2. 2025From Prompts to Power: Measuring the Energy Footprint of LLM Inferencecited by 12
  3. 2024Empirical Measurements of AI Training Power Demand on a GPU-Accelerated Nodecited by 4
  4. 2023EnergAt: Fine-Grained Energy Attribution for Multi-Tenancycited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Position: LLM Inference Should Be Evaluated as Energy-to-Token Production
  2. 2026Understanding Efficiency: Quantization, Batching, and Serving Strategies in LLM Energy Use
  3. 2026Characterization of Request and Token Energy Costs for LLM Inference Workloads on GPU Platforms
  4. 2026Beyond Per-Token Pricing: A Concurrency-Aware Methodology for LLM Infrastructure Cost Estimation
  5. 2026Efficiency and Cost Alignment in Batched LLM Serving via Resource-Fair Scheduling
  6. 2026Towards Resource-Efficient LLMs: End-to-End Energy Accounting of Distillation Pipelines
  7. 2026TokenPowerSandbox: Evidence-Gated CPU-First Screening for Energy-Aware LLM Serving
  8. 2026Token-Budget-Aware Pool Routing for Cost-Efficient LLM Inference