The Quantization Trap: Breaking Linear Scaling Laws in Multi-Hop Reasoning
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
Neural scaling laws provide a predictable recipe for AI advancement: reducing numerical precision should linearly improve computational efficiency and energy profile (E propto bits). In this paper, we demonstrate that this scaling law breaks in the context of multi-hop reasoning. We reveal a 'quantization trap' where reducing precision from 16-bit to 8/4-bit paradoxically increases net energy consumption while degrading reasoning accuracy. We provide a rigorous theoretical decomposition that attributes this failure to hardware casting overhead, the hidden latency cost of dequantization kernels, which becomes a dominant bottleneck in sequential reasoning chains, as well as to a sequential energy amortization failure. As a result, scaling law breaking is unavoidable in practice. We formalize a Critical Model Scale N^* that predicts when the trap dissolves or deepens as a function of model size, batch size, and hardware configuration, validated across a 120× range (0.6B--72B) on six GPU architectures. Our findings suggest that the industry's "smaller-is-better" heuristic is mathematically counterproductive for complex reasoning tasks.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 18 of its 36 references that are in the map. First posted on arXiv 14 Feb 2026.
Builds on
18 of 36 references.
Closest ancestors · 2
Same direction, newest first
- 2025Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models7 Apr
- 2025Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoning6 Jan
Other references · 16
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
- 2022Training Compute-Optimal Large Language Modelscited by 2,927
- 2019The Curious Case of Neural Text Degenerationcited by 2,881
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
- 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
- 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 567
- 2022Efficiently Scaling Transformer Inferencecited by 529
- 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 354
- 2023How Language Model Hallucinations Can Snowballcited by 310
4 more
- 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 255
- 2020Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learningcited by 246
- 2020The De-democratization of AI: Deep Learning and the Compute Divide in Artificial Intelligence Researchcited by 67
- 2025Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latencycited by 32
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Not All Bits Are Equal: Scale-Dependent Memory Optimization Strategies for Reasoning Models
- 2026Quantization Inflates Reasoning: Token Inflation as a Hidden Cost of Low-Bit Reasoning Models
- 2026Scaling Reasoning Hop Exposes Weaknesses: Demystifying and Improving Hop Generalization in Large Language Models
- 2026Understanding Inference Scaling for LLMs: Bottlenecks, Trade-offs, and Performance Principles
- 2026ReSET: Accurate Latency-Critical NVFP4 Reasoning via Step-Aware Temperature Scaling
- 2026Quantized Reasoning Models Think They Need to Think Longer, but They Do Not
- 2026Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recovery
- 2025Cognitive Load Limits in Large Language Models: Benchmarking Multi-Hop Reasoning