mlsanityData to 14 Oct 2026

When Flat Minima Fail: Characterizing INT4 Quantization Collapse After FP32 Convergence

arXiv 2604.15167 ↗First posted 16 Apr 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Post-training quantization (PTQ) assumes that a well-converged model is a quantization-ready model. We show this assumption fails in a structured, measurable, and previously uncharacterized way. Using a calibration-free per-group INT4 probe applied to all 154 publicly available Pythia-160m training checkpoints, we identify a three-phase divergence structure: a rapid-learning phase where both FP32 perplexity and quantization robustness improve together, a meta-stable plateau lasting roughly 70,000 steps where FP32 perplexity stagnates but INT4 gap remains bounded, and an explosive divergence phase where the INT4 gap compounds from 11% to 517% while FP32 perplexity barely moves. Critically, this divergence begins not when the learning rate starts decaying, but precisely when FP32 perplexity converges a finer-grained onset predictor that implies post-convergence weight updates, rather than decay magnitude alone, are the proximate cause. We further show that INT8 quantization is entirely immune throughout all three phases, constraining the mechanism to the coarseness of the 16-level INT4 grid specifically, and rule out weight outlier accumulation as the mechanism via direct kurtosis measurement. Finally, we conduct a controlled fork experiment from the pre-divergence checkpoint comparing three learning rate schedules (cosine continuation, SGDR warm restarts, and our proposed Oscillatory Lock-In) across nine independent runs. SGDR uniformly accelerates divergence (0/9 pairwise wins against cosine), while OLI's settled cool phases reduce the INT4 gap by 2.2 percentage points on average (t = -5.46, p < 0.0001), demonstrating that schedule amplitude calibration, not oscillation alone, determines whether perturbation helps or hurts. Our code, probe implementation, and all 154-checkpoint audit results are released publicly.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 17 references that are in the map. First posted on arXiv 16 Apr 2026.

Builds on

15 of 17 references.

References · 15

  1. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  3. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  4. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  5. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  6. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  7. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  8. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  9. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  10. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  11. 2021A White Paper on Neural Network Quantizationcited by 472
  12. 2021I-BERT: Integer-only BERT Quantizationcited by 255
3 more
  1. 2025Scaling Law for Quantization-Aware Trainingcited by 17
  2. 2025Outlier-Safe Pre-Training for Robust 4-Bit Quantization of Large Language Modelscited by 13
  3. 2021HERO: Hessian-Enhanced Robust Optimization for Unifying and Improving Generalization and Quantization Performancecited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026From Signal Degradation to Computation Collapse: Uncovering the Two Failure Modes of LLM Quantization
  2. 2026QuantForge: Discovering Residual Decompositions for MXFP4 Post-Training Quantization
  3. 2026Pretraining large language models with MXFP4 on Native FP4 Hardware
  4. 2026Global Ranks Survive, Selected Heads Shift: BOS-Sink Topology under 4-bit Weight-Only Quantization
  5. 2025When Less is More: 8-bit Quantization Improves Continual Learning in Large Language Models
  6. 2026StableQAT: Stable Quantization-Aware Training at Ultra-Low Bitwidths
  7. 2026RAMP: Robust Adaptive Mixed-Precision Quantization for Edge CPU Vision Models
  8. 2026The Curse and Blessing of Mean Bias in FP4-Quantized LLM Training