Calibrated e-CUSUM Decoding for Quantized Reasoning Models: Why Token Log-Probability Is the Wrong Observable for Decoding Monitors
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
Low-bit quantization makes small reasoning models inexpensive to deploy but can degrade their chains of thought. This motivates decoder-side monitors that intervene when generation becomes unreliable. We show that a natural candidate, the centered token log-probability increment log p(wt)+Ht, is the wrong observable for this purpose. Under the model's own sampling law it is a mean-zero martingale by construction, so it measures sampling self-consistency rather than trajectory health and is nearly silent during confident repetition, where both log p(wt) and entropy are close to zero. We introduce a training-free decoding controller that combines (i) a degeneration-aware alarm score fusing token uncertainty with explicit verbatim repetition and (ii) a calibrated e-process-inspired sequential detector. The raw product process is Ville-valid under a conditional-mean null, while the deployed CUSUM-floored statistic is treated as an empirical change detector because the score is history-dependent and autocorrelated. On GSM8K with DeepSeek-R1-Distill-Qwen-1.5B in FP16 and INT4, calibration turns a monitor that fires on 93--95% of generations into a selective detector of failing traces (φ ≈ 0.3, precision ≈ 0.6 against a 0.38 base rate). In this pilot, the controller reduces measured verbatim-degeneration signals and yields a positive but statistically inconclusive INT4 accuracy change from 63% to 69% (paired McNemar p=0.18, n=100), at a 28% token-budget cost. We also find that non-termination, rather than looping, is the dominant failure mode on GSM8K. The main contribution is methodological: an explanation of why centered token log-probability is inadequate for decoder monitoring and a calibrated, cautiously evaluated replacement.
Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 22 of its 27 references that are in the map. First posted on arXiv 13 Jul 2026.
Builds on
22 of 27 references.
Closest ancestors · 9
Same direction, newest first
- 2026Quantized Reasoning Models Think They Need to Think Longer, but They Do Not29 May
- 2026Mitigating Overthinking in Large Reasoning Language Models via Reasoning Path Deviation Monitoring15 Mar
- 2026Circular Reasoning: Understanding Self-Reinforcing Loops in Large Reasoning Models9 Jan
- 2025Deep Think with Confidence21 Aug
- 2025Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models3 Jul
- 2025Quantization Meets Reasoning: Exploring and Mitigating Degradation of Low-Bit LLMs in Mathematical Reasoning16 May
- 2025Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models7 Apr
- 2025Step Back to Leap Forward: Self-Backtracking for Boosting Reasoning of Language Models6 Feb
- 2025Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs30 Jan
Other references · 13
- 2021Training Verifiers to Solve Math Word Problemscited by 9,916
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,657
- 2023Let's Verify Step by Stepcited by 3,672
- 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,388
- 2019The Curious Case of Neural Text Degenerationcited by 2,893
- 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoningcited by 162
- 2022Game-theoretic statistics and safe anytime-valid inferencecited by 149
- 2024Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputscited by 148
- 2023Repetition In Repetition Out: Towards Understanding Neural Text Degeneration from the Data Perspectivecited by 39
- 2022E-detectors: a nonparametric framework for sequential change detectioncited by 30
- 2025WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingalescited by 14
- 2023Penalty Decoding: Well Suppress the Self-Reinforcement Effect in Open-Ended Text Generationcited by 13
1 more
- 2024Adaptive Conformal Inference by Bettingcited by 12
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026CUSUM-Shaped Inference-Time Monitoring and Targeted Re-Decoding for Quantized Small Language Model Reasoning
- 2026BitCal-TTS: Bit-Calibrated Test-Time Scaling for Quantized Reasoning Models
- 2026ReSET: Accurate Latency-Critical NVFP4 Reasoning via Step-Aware Temperature Scaling
- 2026Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recovery
- 2026Quantization Inflates Reasoning: Token Inflation as a Hidden Cost of Low-Bit Reasoning Models
- 2026Spend Bytes on Breadth: Precision-Count Trade-offs for Decode-Time KV Compression in Long Chain-of-Thought Reasoning
- 2025ThinkBrake: Efficient Reasoning via Log-Probability Margin Guided Decoding
- 2026Train Where the Quantized Model Goes: On-Policy Distillation for Low-Bit Reasoning