mlsanityData to 7 Oct 2026

The Detection-Extraction Gap: Models Know the Answer Before They Can Say It

arXiv 2604.06613 ↗First posted 8 Apr 2026cs.CL cs.AI cs.ITcited by 2

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

also in 1 more
Abstract (arXiv)

Modern reasoning models continue generating long after the answer is already determined. Across five model configurations, two families, and three benchmarks, we find that 52--88% of chain-of-thought tokens are produced after the answer is recoverable from a partial prefix. This post-commitment generation reveals a structural phenomenon: the detection-extraction gap. Free continuations from early prefixes recover the correct answer even at 10% of the trace, while forced extraction fails on 42% of these cases. The answer is recoverable from the model state, yet prompt-conditioned decoding fails to extract it. We formalize this mismatch via a total-variation bound between free and forced continuation distributions, yielding quantitative estimates of suffix-induced shift. Exploiting this asymmetry, we propose Black-box Adaptive Early Exit (BAEE), which uses free continuations for both detection and extraction, truncating 70--78% of serial generation while improving accuracy by 1--5pp across all models. For thinking-mode models, early exit prevents post-commitment overwriting, yielding gains of up to 5.8pp; a cost-optimized variant achieves 68--73% reduction at a median of 9 API calls. Code is available at https://github.com/EdWangLoDaSc/know2say.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 21 of its 28 references that are in the map. First posted on arXiv 8 Apr 2026.

Builds on

21 of 28 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought5 Mar
  2. 2026Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answers2 Mar
  3. 2025Monitoring Monitorability20 Dec
  4. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning18 Sep
  5. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting7 May
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar

Other references · 15

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  4. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  5. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  6. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 564
  7. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  8. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 302
  9. 2025Dynamic Early Exit in Reasoning Modelscited by 217
  10. 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verificationcited by 154
  11. 2025The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Taskscited by 150
  12. 2023EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelismcited by 69
3 more
  1. 2024Combining Confidence Elicitation and Sample-based Methods for Uncertainty Quantification in Misinformation Mitigationcited by 19
  2. 2025Text2Grad: Reinforcement Learning from Natural Language Feedbackcited by 18
  3. 2024Confidence Estimation for LLM-Based Dialogue State Trackingcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Know When To Fold 'Em: Token-Efficient LLM Synthetic Data Generation via Multi-Stage In-Flight Rejectioncited by 0
  2. 2026Decodable In-Context State and Model Output Across Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Models
  2. 2026Stop When Reasoning Converges: Semantic-Preserving Early Exit for Reasoning Models
  3. 2026Large Language Models Decide Early and Explain Later
  4. 2026Rethinking Dense Sequential Chains: Reasoning Language Models Can Extract Answers from Sparse, Order-Shuffling Chain-of-Thoughts
  5. 2026When Confidence Rises Too Early: Detecting Shortcut Reasoning via Premature Answer Commitment
  6. 2026When Reasoning Traces Become Performative: Step-Level Evidence that Chain-of-Thought Is an Imperfect Oversight Channel
  7. 2026</think> Doesn't Stop Reasoning: Analysis of Spurious CoT Termination
  8. 2026Probing the Trajectories of Reasoning Traces in Large Language Models