mlsanityData to 14 Oct 2026

TokenPowerSandbox: Evidence-Gated CPU-First Screening for Energy-Aware LLM Serving

arXiv 2608.18149 ↗First posted 11 Aug 2026cs.AR cs.AI cs.DCcited by 0

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

Energy-aware LLM serving requires comparing configurations under realistic request shapes, yet exhaustive target-GPU profiling is costly and a cheap predictor can be dangerously confident outside its measured scope. We present TokenPowerSandbox, an evidence-gated workflow that combines an interpretable CPU-resident projector, short target-GPU probes, full-workload verification, and tamper-evident freeze-before-measurement provenance. On one NVIDIA H100 80GB serving Qwen2.5-7B-Instruct with vLLM, three anchor repeats and six development workloads calibrate workload transfer. The same frozen model is evaluated on a blind holdout and a separately predeclared no-refit confirmation totaling 51 post-freeze runs. Energy MAPE is 6.23% and 7.35%, with Spearman rank correlations of 0.976 and 0.933. However, a predeclared TTFT gate passes at concurrency four (9.27% MAPE) and triggers abstention below four (64.80%), showing why energy accuracy cannot certify latency.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 14 of its 20 references that are in the map. First posted on arXiv 11 Aug 2026.

Builds on

14 of 20 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures10 Apr
  2. 2026BOute: Cost-Efficient LLM Serving with Heterogeneous LLMs and GPUs via Multi-Objective Bayesian Optimization11 Feb
  3. 2024Offline Energy-Optimal LLM Serving: Workload-Based Energy Models for LLM Inference on Heterogeneous Systems4 Jul
  4. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve4 Mar

Other references · 10

  1. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 470
  2. 2024Vidur: A Large-Scale Simulation Framework For LLM Inferencecited by 117
  3. 2024DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiencycited by 112
  4. 2025Energy Considerations of Large Language Model Inference and Efficiency Optimizationscited by 55
  5. 2025The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimizationcited by 33
  6. 2025TokenPowerBench: Benchmarking the Power Consumption of LLM Inferencecited by 22
  7. 2024LLMCO2: Advancing Accurate Carbon Footprint Prediction for LLM Inferencescited by 19
  8. 2025From Prompts to Power: Measuring the Energy Footprint of LLM Inferencecited by 12
  9. 2025FIXME: Towards End-to-End Benchmarking of LLM-Aided Design Verificationcited by 9
  10. 2026From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Where Does the Energy Go? Profiling LLM Agent Inference on Blackwell GPUs ↗
  2. 2026The 1/W Law: An Analytical Study of Context-Length Routing Topology and GPU Generation Gains for LLM Inference Energy Efficiency
  3. 2026Characterization of Request and Token Energy Costs for LLM Inference Workloads on GPU Platforms
  4. 2026Hydra: Phase-Aware Workload Characterization of LLM Inference across Edge SoC Generations, Backends, and Quantization Levels
  5. 2026Energy-Aware Scheduling for Serverless LLM Serving on Shared GPUs
  6. 2026Position: LLM Inference Should Be Evaluated as Energy-to-Token Production
  7. 2026Phase-Decoupled, Model-Calibrated Power Control for Disaggregated LLM Serving
  8. 2026Measurement-Driven Diagnosis and Mitigation of Host-CPU Co-location Interference in Single-GPU LLM Serving on a Multi-GPU Server ↗