mlsanityData to 7 Oct 2026

The Coupling Tax: How Shared Token Budgets Undermine Visible Chain-of-Thought Under Fixed Output Limits

arXiv 2605.07686 ↗First posted 8 May 2026cs.LGcited by 0

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Chain-of-thought reasoning is often treated as a monotone way to improve language-model accuracy by letting a model think longer. We identify a countervailing effect, the coupling tax: when reasoning traces and final answers share one output-token budget, long traces can crowd out the answer they are meant to support. Across GSM8K, MATH-500, and five BIG-Bench Hard tasks with Qwen3 models at three scales, non-thinking mode matches or outperforms thinking mode on GSM8K and MATH-500 at every budget up to 2048 tokens, while harder tasks shift the crossover to larger budgets. We derive a truncation-waste decomposition, Accthink(b)=αc FL(b)+αt(1-FL(b)), that predicts this crossover from chain-length and accuracy statistics and explains inverse scaling within the Qwen family. A DeepSeek-R1-Distill-Llama-8B replication shows the same pattern under a different thinking interface. As a mitigation, split-budget generation decouples reasoning and answer budgets; on full MATH-500, IRIS reaches 74.0% accuracy, a strengthened extraction variant reaches 78.8%, and a fixed non-oracle SC+IRIS gate reaches 83.6%. The results show that test-time reasoning should be evaluated as a budget-allocation problem, not only as a question of whether longer traces are available.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 23 of its 28 references that are in the map. First posted on arXiv 8 May 2026.

Builds on

23 of 28 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them17 Oct
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 20

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2023Let's Verify Step by Stepcited by 3,598
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  7. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  8. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  9. 2025s1: Simple test-time scalingcited by 1,311
  10. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  11. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  12. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
8 more
  1. 2018Universal Transformerscited by 613
  2. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 564
  3. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 479
  4. 2022Confident Adaptive Language Modelingcited by 297
  5. 2025Reasoning Models Can Be Effective Without Thinkingcited by 182
  6. 2023Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decodingcited by 80
  7. 2025SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoningcited by 47
  8. 2025Scalable Chain of Thoughts via Elastic Reasoningcited by 32

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Token Budget Saturation and Mechanistic Early Detection of Reasoning Non-Convergence in Chain-of-Thought Models
  2. 2026Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Models
  3. 2026Mechanistic Evidence for Faithfulness Decay in Chain-of-Thought Reasoning
  4. 2026When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling
  5. 2026When Reasoning Traces Become Performative: Step-Level Evidence that Chain-of-Thought Is an Imperfect Oversight Channel
  6. 2026Brief Is Better: Non-Monotonic Chain-of-Thought Budget Effects in Function-Calling Language Agents
  7. 2025Increasing the Thinking Budget is Not All You Need
  8. 2025Let Me Think! A Long Chain-of-Thought Can Be Worth Exponentially Many Short Ones