mlsanityData to 14 Oct 2026

From Signal Degradation to Computation Collapse: Uncovering the Two Failure Modes of LLM Quantization

arXiv 2604.19884 ↗First posted 21 Apr 2026cs.CL cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Post-Training Quantization (PTQ) is critical for the efficient deployment of Large Language Models (LLMs). While 4-bit quantization is widely regarded as an optimal trade-off, reducing the precision to 2-bit usually triggers a catastrophic ``performance cliff.'' It remains unclear whether the underlying mechanisms differ fundamentally. Consequently, we conduct a systematic mechanistic analysis, revealing two qualitatively distinct failure modes: Signal Degradation, where the computational patterns remain intact but information precision is impaired by cumulative error; and Computation Collapse, where key components fail to function, preventing correct information processing and destroying the signal in the early layers. Guided by this diagnosis, we conduct mechanism-aware interventions, demonstrating that targeted, training-free repair can mitigate Signal Degradation, but remains ineffective for Computation Collapse. Our findings provide a systematic diagnostic framework for PTQ failures and suggest that addressing Computation Collapse requires structural reconstruction rather than mere compensation.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 22 of its 32 references that are in the map. First posted on arXiv 21 Apr 2026.

Builds on

22 of 32 references.

References · 22

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2023Mistral 7Bcited by 3,164
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  7. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  8. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  9. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  10. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  11. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  12. 2024How to use and interpret activation patchingcited by 214
10 more
  1. 2024Catastrophic Failure of LLM Unlearning via Quantizationcited by 81
  2. 2024Evaluating Quantized Large Language Modelscited by 77
  3. 2025Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Modelscited by 73
  4. 2024A Comprehensive Evaluation of Quantization Strategies for Large Language Modelscited by 48
  5. 2025Why Do Some Inputs Break Low-Bit LLM Quantization?cited by 9
  6. 2025Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMscited by 9
  7. 2025Towards Superior Quantization Accuracy: A Layer-sensitive Approachcited by 8
  8. 2024EoRA: Fine-tuning-free Compensation for Compressed LLM with Eigenspace Low-Rank Approximationcited by 7
  9. 2025Towards Understanding and Improving Refusal in Compressed Models via Mechanistic Interpretabilitycited by 7
  10. 2025Exploring Layer-wise Information Effectiveness for Post-Training Quantization in Small Language Modelscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Beyond Activation Alignment:The Alignment-Diversity Tradeoff in Task-Aware LLM Quantizationcited by 1
  2. 2026SPECTRA: Pushing the KV Cache Beyond the 2-Bit Cliff via Spectral Transform Codingcited by 1
  3. 2026Beyond Independent Optimization: Compression, MoE Routing, and Quantization Interactions in Multimodal Edge Intelligencecited by 0
  4. 2026Quantization Degradation in Large Language Models: A Signal-Noise Perspectivecited by 0
  5. 2026Saying, Not Knowing: Aggressively GGUF-Quantized Small Language Models Still Write Rare Words They Can No Longer Definecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026The Structure of Quantization Damage in LLMs: Why the Next Bit Should Be Spent Globally
  2. 2024Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokens
  3. 2026SPEAR: A System for Post-Quantization Error-Adaptive Recovery Enabling Efficient Low-Bit LLM Serving
  4. 2026Accuracy is Not Enough: A Divergence-Based Approach to Evaluate Fidelity Loss in Quantized LLMs
  5. 2024What Makes Quantization for Large Language Models Hard? An Empirical Study from the Lens of Perturbation
  6. 2026Fitting Is Not Enough: Smoothness in Extremely Quantized LLMs
  7. 2026Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts
  8. 2026Quantization-Aware Healing: A Practical Recipe for Recovering Compressed, 4-Bit LLMs