# TokenPowerSandbox: Evidence-Gated CPU-First Screening for Energy-Aware LLM Serving

arXiv: https://arxiv.org/abs/2608.18149 · HTML: https://mlsanity.com/paper/2608.18149/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving.
- Builds on 14 of its 20 references that are in the map.
- First posted on arXiv 11 Aug 2026.

## Placed in

- Efficient LLM inference and serving / [Serving systems and request scheduling](https://mlsanity.com/inference/serving-systems-and-request-scheduling/)

## Builds on (most cited first, 14 of 14)

- [Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve](https://mlsanity.com/paper/2403.02310/) (2024)
- [BOHB: Robust and Efficient Hyperparameter Optimization at Scale](https://mlsanity.com/paper/1807.01774/) (2018)
- [Vidur: A Large-Scale Simulation Framework For LLM Inference](https://mlsanity.com/paper/2405.05465/) (2024)
- [DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency](https://mlsanity.com/paper/2408.00741/) (2024)
- [Energy Considerations of Large Language Model Inference and Efficiency Optimizations](https://mlsanity.com/paper/2504.17674/) (2025)
- [The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimization](https://mlsanity.com/paper/2505.06371/) (2025)
- [Offline Energy-Optimal LLM Serving: Workload-Based Energy Models for LLM Inference on Heterogeneous Systems](https://mlsanity.com/paper/2407.04014/) (2024)
- [TokenPowerBench: Benchmarking the Power Consumption of LLM Inference](https://mlsanity.com/paper/2512.03024/) (2025)
- [LLMCO2: Advancing Accurate Carbon Footprint Prediction for LLM Inferences](https://mlsanity.com/paper/2410.02950/) (2024)
- [From Prompts to Power: Measuring the Energy Footprint of LLM Inference](https://mlsanity.com/paper/2511.05597/) (2025)
- [FIXME: Towards End-to-End Benchmarking of LLM-Aided Design Verification](https://mlsanity.com/paper/2507.04276/) (2025)
- [BOute: Cost-Efficient LLM Serving with Heterogeneous LLMs and GPUs via Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2602.10729/) (2026)
- [Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures](https://mlsanity.com/paper/2604.09048/) (2026)
- [From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs](https://mlsanity.com/paper/2607.26571/) (2026)

## Abstract (arXiv)

Energy-aware LLM serving requires comparing configurations under realistic request shapes, yet exhaustive target-GPU profiling is costly and a cheap predictor can be dangerously confident outside its measured scope. We present TokenPowerSandbox, an evidence-gated workflow that combines an interpretable CPU-resident projector, short target-GPU probes, full-workload verification, and tamper-evident freeze-before-measurement provenance. On one NVIDIA H100 80GB serving Qwen2.5-7B-Instruct with vLLM, three anchor repeats and six development workloads calibrate workload transfer. The same frozen model is evaluated on a blind holdout and a separately predeclared no-refit confirmation totaling 51 post-freeze runs. Energy MAPE is 6.23% and 7.35%, with Spearman rank correlations of 0.976 and 0.933. However, a predeclared TTFT gate passes at concurrency four (9.27% MAPE) and triggers abstention below four (64.80%), showing why energy accuracy cannot certify latency.