mlsanityData to 7 Oct 2026

CUSUM-Shaped Inference-Time Monitoring and Targeted Re-Decoding for Quantized Small Language Model Reasoning

arXiv 2607.20129 ↗First posted 22 Jul 2026cs.AIcited by 1

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Quantized small reasoning models can enter repetitive or otherwise unproductive trajectories, yet standard decoding does not adapt to the trajectory as it unfolds. We study MGT-B, a fixed, weight-preserving controller that converts overlapping windows of uncertainty, repetition, and local-change features into position-conditional empirical tail probabilities. It accumulates mixture betting factors with a CUSUM-shaped reset, and, after an alarm, restores a coherent earlier token and key-value-cache state before constrained re-decoding. On MATH-500, a paired three-seed evaluation over 1,500 generations per method raises exact-normalized accuracy from 54.73% for vanilla decoding to 56.40% (+1.67 percentage points; problem-clustered bootstrap 95% CI [+0.47, +2.80]), while a prospectively profiled random-intervention control reaches 54.60%. The gain is positive in all three seeds and costs 5.14% more sampled tokens. Seed-0 ablations show that rollback alone does not explain the result and that an isolated repetition penalty is harmful. Five-sample self-consistency reaches 70.0% but uses about 4.84x as many tokens as MGT-B. On the harder, non-overlapping Omni-MATH evaluation, however, MGT-B obtains 16.60% versus 16.67% for vanilla (-0.07 points; clustered 95% CI [-0.33, +0.20]) with 2.10% more sampled tokens. Thus, MGT-B provides a modest, reproducible local improvement on MATH-500 in the studied configuration, but the effect does not transfer to Omni-MATH and should not be interpreted as a general improvement in mathematical reasoning.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 27 of its 33 references that are in the map. First posted on arXiv 22 Jul 2026.

Builds on

27 of 33 references.

Closest ancestors · 16

Same direction, newest first

  1. 2026Quantized Reasoning Models Think They Need to Think Longer, but They Do Not29 May
  2. 2026The Shape of Overthinking: Backtracking Bursts in Long Reasoning Traces27 May
  3. 2026Beyond the Frontier: Stochastic Backtracking for Efficient Test-Time Scaling24 May
  4. 2026Stop When Reasoning Converges: Semantic-Preserving Early Exit for Reasoning Models17 May
  5. 2026Mitigating Overthinking in Large Reasoning Language Models via Reasoning Path Deviation Monitoring15 Mar
  6. 2026Circular Reasoning: Understanding Self-Reinforcing Loops in Large Reasoning Models9 Jan
  7. 2025The Evolution of Thought: Tracking LLM Overthinking via Reasoning Dynamics Analysis25 Aug
  8. 2025Deep Think with Confidence21 Aug
  9. 2025Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models3 Jul
  10. 2025Quantization Meets Reasoning: Exploring and Mitigating Degradation of Low-Bit LLMs in Mathematical Reasoning16 May
  11. 2025Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models7 Apr
  12. 2025Step Back to Leap Forward: Self-Backtracking for Boosting Reasoning of Language Models6 Feb
4 more
  1. 2025Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs30 Jan
  2. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  3. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May
  4. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar

Other references · 11

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  2. 2023Let's Verify Step by Stepcited by 3,598
  3. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  4. 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoningcited by 160
  5. 2024Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantificationcited by 159
  6. 2022Game-theoretic statistics and safe anytime-valid inferencecited by 145
  7. 2023Repetition In Repetition Out: Towards Understanding Neural Text Degeneration from the Data Perspectivecited by 39
  8. 2022E-detectors: a nonparametric framework for sequential change detectioncited by 27
  9. 2023Penalty Decoding: Well Suppress the Self-Reinforcement Effect in Open-Ended Text Generationcited by 13
  10. 2025WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingalescited by 13
  11. 2024Adaptive Conformal Inference by Bettingcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Calibrated e-CUSUM Decoding for Quantized Reasoning Models: Why Token Log-Probability Is the Wrong Observable for Decoding Monitors
  2. 2026Selective Regenerative Decoding: Trajectory-Level Intervention for Inference-Time Reasoning
  3. 2026BitCal-TTS: Bit-Calibrated Test-Time Scaling for Quantized Reasoning Models
  4. 2025Less is More: Improving LLM Reasoning with Minimal Test-Time Intervention
  5. 2026Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recovery
  6. 2025A*-Decoding: Token-Efficient Inference Scaling
  7. 2026Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts
  8. 2025Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs