mlsanityData to 14 Oct 2026

The Energy Cost of Reasoning: Analyzing Energy Usage in LLMs with Test-time Compute

arXiv 2505.14733 ↗First posted 20 May 2025cs.LG cs.AIcited by 15

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Scaling large language models (LLMs) has driven significant advancements, yet it faces diminishing returns and escalating energy demands. This work explores how test-time compute (TTC) can serve as an energy-efficient complement to conventional scaling strategies by allocating additional computational resources at inference time rather than during training. Specifically, we investigate whether employing TTC can achieve superior accuracy-energy trade-offs compared to simply increasing model size. Our empirical analysis reveals that TTC surpasses traditional model scaling in accuracy/energy efficiency, with notable gains in tasks demanding complex reasoning rather than mere factual recall. Further, we identify a critical interaction between TTC performance and output sequence length, demonstrating that strategically adjusting compute resources at inference time according to query complexity can substantially enhance efficiency. Our findings advocate for TTC as a promising direction, enabling more sustainable, accurate, and adaptable deployment of future language models.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 46 of its 64 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

46 of 64 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Large Language Models have Intrinsic Self-Correction Ability21 Jun
  2. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  3. 2023Large Language Models Cannot Self-Correct Reasoning Yet3 Oct
  4. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May
  5. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar

Other references · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  7. 2020Scaling Laws for Neural Language Modelscited by 6,586
  8. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  9. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  10. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  11. 2024Qwen2.5 Technical Reportcited by 4,942
  12. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
29 more
  1. 2021Program Synthesis with Large Language Modelscited by 3,637
  2. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  3. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  4. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  5. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  6. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  7. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  8. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  9. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 618
  10. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
  11. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 596
  12. 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 407
  13. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 349
  14. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 273
  15. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  16. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 240
  17. 2022Will we run out of data? Limits of LLM scaling based on human-generated datacited by 195
  18. 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 191
  19. 2022The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrinkcited by 184
  20. 2023Speculative Decoding with Big Little Decodercited by 164
  21. 2025CodeElo: Benchmarking Competition-level Code Generation of LLMs with Human-comparable Elo Ratingscited by 115
  22. 2023Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Clustercited by 109
  23. 2024NanoFlow: Towards Optimal Large Language Model Serving Throughputcited by 90
  24. 2023SkipDecode: Autoregressive Skip Decoding with Batching and Caching for Efficient LLM Inferencecited by 89
  25. 2023EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelismcited by 69
  26. 2024Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inferencecited by 61
  27. 2023Rewarding Chatbots for Real-World Engagement with Millions of Userscited by 59
  28. 2025Energy Considerations of Large Language Model Inference and Efficiency Optimizationscited by 55
  29. 2025Unveiling Environmental Impacts of Large Language Model Serving: A Functional Unit Viewcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 8.

15 papers.

Sort:

By area:

  1. 2025Murakkab: Resource-Efficient Agentic Workflow Orchestration in Cloud Platformscited by 18
  2. 2025Energy Use of AI Inference, Efficiency Pathways, and Test-Time Scalingcited by 13
  3. 2025Optimal Stopping vs Best-of-N for Inference Time Optimizationcited by 7
  4. 2026From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AIcited by 6
  5. 2026Understanding Efficiency: Quantization, Batching, and Serving Strategies in LLM Energy Usecited by 4
  6. 2025Explicit Reasoning Makes Better Judges: A Systematic Study on Accuracy, Efficiency, and Robustnesscited by 3
  7. 2026Multi-Agent Reasoning Improves Compute Efficiency: Pareto-Optimal Test-Time Scalingcited by 3
  8. 2026Characterization of Request and Token Energy Costs for LLM Inference Workloads on GPU Platformscited by 2
  9. 20253D Optimization for AI Inference Scaling: Balancing Accuracy, Cost, and Latencycited by 1
  10. 2025Verifier Threshold: An Efficient Test-Time Scaling Approach for Image Generationcited by 1
  11. 2025Compute-Accuracy Pareto Frontiers for Open-Source Reasoning Large Language Modelscited by 1
  12. 2026Small Talk, Big Impact: The Energy Cost of Thanking AIcited by 1
3 more
  1. 2026Balancing Sustainability And Performance: The Role Of Small-Scale LLMs In Agentic Artificial Intelligence Systemscited by 0
  2. 2026Cut Your Losses! Learning to Prune Paths Early for Efficient Parallel Reasoningcited by 0
  3. 2026Quantifying the Relationship Between Clinical Safety and Environmental Impact in Therapeutic LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Test-Time Compute Strategies: Advocating Energy-per-Token in LLM Inference
  2. 2025The Art of Scaling Test-Time Compute for Large Language Models
  3. 2025TokenPowerBench: Benchmarking the Power Consumption of LLM Inference
  4. 2026ThinkBooster: A Unified Framework for Seamless Test-Time Scaling of LLM Reasoning
  5. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs
  6. 2025Are We Scaling the Right Thing? A System Perspective on Test-Time Scaling
  7. 2025Faster and Better LLMs via Latency-Aware Test-Time Scaling
  8. 2025An Empirical Study of LLM Reasoning Ability Under Strict Output Length Constraint