The Reasoning Tax: Token Economics of LLM Reasoning Across Task Types and Deployment Contexts
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
Accuracy-only benchmarking of reasoning-capable large language models misses a central deployment question: when do extended thinking tokens earn their cost? We introduce the Token Economy Score (TES), a marginal benchmarking metric that measures the accuracy gain of a reasoning model over a non-reasoning baseline, normalized by the generated-token multiplier. We define paired and approximated TES variants for model families with reasoning toggles and frontier models without direct non-reasoning counterparts. We then conduct an empirical benchmarking analysis across 151 model-benchmark evaluation runs on seven benchmarks spanning mathematics, code generation, science reasoning, instruction following, expert knowledge, knowledge recall, and research-level physics. The analysis examines three deployment-facing dimensions: which task structures yield positive marginal reasoning efficiency, how increasing reasoning effort changes TES within model families, and how deployment context changes economic viability. Results show that task structure predicts reasoning efficiency better than nominal difficulty: sequential inferencechain tasks such as AIME 2025 and LiveCodeBench show high TES, while knowledge-recall tasks such as MMLU-Pro show low TES despite their difficulty. We also find systematic diminishing returns at higher reasoning effort levels, including cases where additional thinking reduces accuracy. Finally, Reasoning Cost Share (RCS) shows that inference spend is often dominated by internal thinking, while Deployment Cost Multiplier (DCM) shows how on-premises deployment can change the economics of otherwise costly reasoning workloads. These findings support a benchmarking-driven model-selection rule: enable reasoning selectively by task type, effort level, and deployment context rather than treating it as a universally beneficial mode.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 15 of its 25 references that are in the map. First posted on arXiv 26 Aug 2026.
Builds on
15 of 25 references.
Closest ancestors · 1
Same direction, newest first
Other references · 14
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2025Qwen3 Technical Reportcited by 8,820
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
- 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
- 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 297
- 2025Generalizing Verifiable Instruction Followingcited by 194
- 2025GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Taskscited by 158
- 2026APEX-Agentscited by 32
- 2025Probing the Critical Point (CritPt) of AI Reasoning: a Frontier Physics Research Benchmarkcited by 21
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies
- 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models
- 2026On the Token Value Inequality in Efficient Reasoning
- 2025ReasonBENCH: Benchmarking the (In)Stability of LLM Reasoning
- 2026Beyond Accuracy: Decomposing the Reasoning Efficiency of LLMs
- 2026When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling
- 2025Beyond Chains of Thought: Benchmarking Latent-Space Reasoning Abilities in Large Language Models
- 2025The Ouroboros of Benchmarking: Reasoning Evaluation in an Era of Saturation