# Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recovery

arXiv: https://arxiv.org/abs/2606.02011 · HTML: https://mlsanity.com/paper/2606.02011/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 28 of its 40 references that are in the map.
- First posted on arXiv 1 Jun 2026.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 28 of 28)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies](https://mlsanity.com/paper/2101.02235/) (2021)
- [Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models](https://mlsanity.com/paper/2503.16419/) (2025)
- [Dynamic Early Exit in Reasoning Models](https://mlsanity.com/paper/2504.15895/) (2025)
- [WinoGrande: An Adversarial Winograd Schema Challenge at Scale](https://mlsanity.com/paper/1907.10641/) (2019)
- [Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models](https://mlsanity.com/paper/2504.04823/) (2025)
- [SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning](https://mlsanity.com/paper/2504.07891/) (2025)
- [Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Time](https://mlsanity.com/paper/2504.12329/) (2025)
- [ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models](https://mlsanity.com/paper/2408.08554/) (2024)
- [Efficient Reasoning for LLMs through Speculative Chain-of-Thought](https://mlsanity.com/paper/2504.19095/) (2025)
- [Fast and Efficient 2-bit LLM Inference on GPU: 2/4/16-bit in a Weight Matrix with Asynchronous Dequantization](https://mlsanity.com/paper/2311.16442/) (2023)
- [Self-Steering Language Models](https://mlsanity.com/paper/2504.07081/) (2025)
- [SplitReason: Learning To Offload Reasoning](https://mlsanity.com/paper/2504.16379/) (2025)
- [ML-SpecQD: Multi-Level Speculative Decoding with Quantized Drafts](https://mlsanity.com/paper/2503.13565/) (2025)
- [Quantization Meets Reasoning: Exploring and Mitigating Degradation of Low-Bit LLMs in Mathematical Reasoning](https://mlsanity.com/paper/2505.11574/) (2025)
- [Efficient LLM Collaboration via Planning](https://mlsanity.com/paper/2506.11578/) (2025)
- [TERMINATOR: Learning Optimal Exit Points for Early Stopping in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2603.12529/) (2026)
- [GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling](https://mlsanity.com/paper/2604.18556/) (2026)
- [QuantLRM: Quantization of Large Reasoning Models via Fine-Tuning Signals](https://mlsanity.com/paper/2602.02581/) (2026)
- [HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization](https://mlsanity.com/paper/2605.29843/) (2026)
- [Tandem: Riding Together with Large and Small Language Models for Efficient Reasoning](https://mlsanity.com/paper/2604.23623/) (2026)
- [From Quarter to All: Accelerating Speculative LLM Decoding via Floating-Point Exponent Remapping and Parameter Sharing](https://mlsanity.com/paper/2510.18525/) (2025)
- [Quasar: Quantized Self-Speculative Acceleration for Rapid Inference via Memory-Efficient Verification](https://mlsanity.com/paper/2603.01399/) (2026)

## Built on by (most cited first, 1 of 1)

- [RATIO: Reasoning Analysis and Token-level Inference Optimization for Quantized Reasoning Models](https://mlsanity.com/paper/2609.39801/) (2026)

## Abstract (arXiv)

Large Reasoning Models (LRMs) rely on long reasoning traces, making inference expensive. While low-bit quantization reduces per-token decoding cost, we show that aggressive 2-bit inference can fail to deliver end-to-end speedup because instability in the generation process inflates total token count. Instead of merely lowering answer accuracy, 2-bit quantization often produces much longer traces with repetitive loops, budget exhaustion, delayed commitment, and unclosed reasoning segments. We analyze full reasoning traces of Qwen3 reasoning models across mathematical and commonsense benchmarks and show that accuracy degradation is tightly linked to these process-level failures. To address them, we introduce two lightweight controls: FP16 planning, which gives the 2-bit model a short high-precision outline, and loop rescue, which detects repetitive traces and either commits to an earlier answer or falls back to FP16. On MATH-500, loop rescue improves Qwen3-8B accuracy from 17.2% to 74.2%, while planning plus loop rescue improves Qwen3-32B from 65.0% to 87.2%. Overall, our results show that extreme low-bit reasoning becomes practical when its failures are treated as controllable generation pathologies: with lightweight detection and selective FP16 support, 2-bit inference can recover accuracy while preserving real end-to-end speed. Our code is available at: https://github.com/brain-lab-research/quantized-reasoning.