mlsanityData to 14 Oct 2026

Thinking Costs Tokens: When More Structure is Worth the Price

arXiv 2608.27506 ↗First posted 27 Aug 2026cs.AIcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Adding inference structure to a language model lets it search, verify, and revise, but these actions consume the very budget they are supposed to use well. In this paper, we investigate whether there exists a token-budget threshold, below which the overhead of planning and verification hurts performance and above which it helps. We evaluate two systems on FinQA and TAT-QA financial reasoning tasks, using GPT-5.4 mini across 14 budget tiers ranging from 250 to 42,000 output-equivalent tokens. The first system is a monolith, which is a single LLM call. The second is a verified search architecture that adds planning, label-blind checking, and repair capabilities. We run 1,000 cases for a total of 28,000 completed cells. Both systems score 0% at the two lowest tiers, where neither can fit a complete prompt. At 1,000 tokens, the monolith reaches 18% accuracy while verified search scores near 0%, since the planning overhead leaves no room for an answer. From 1,500 tokens onward, verified search surpasses the monolith and maintains a consistent advantage, reaching approximately 44% at the highest tiers while the monolith reaches approximately 40%. The crossover occurs between 1,000 and 1,500 output-equivalent tokens, confirmed by a strict intersection-union test (p ≤ 0.001 at both endpoints).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 12 of its 12 references that are in the map. First posted on arXiv 27 Aug 2026.

Builds on

12 of 12 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  2. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May
  3. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar

Other references · 9

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  6. 2023Let's Verify Step by Stepcited by 3,789
  7. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 562
  8. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 408
  9. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 240

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Plan and Budget: Effective and Efficient Test-Time Scaling on Reasoning Large Language Models
  2. 2026The Shadow Price of Reasoning: Economic Perspective on Optimal Budget Allocation for LLMs
  3. 2026The Reasoning Tax: Token Economics of LLM Reasoning Across Task Types and Deployment Contexts
  4. 2024Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies
  5. 2026On the Token Value Inequality in Efficient Reasoning
  6. 2026Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements
  7. 2024Token-Budget-Aware LLM Reasoning
  8. 2026Conformal Thinking: Risk Control for Reasoning on a Compute Budget