mlsanityData to 14 Oct 2026

Max-Window Scale Estimation for Near-Lossless HiF8 W8A8 Quantization-Aware Training

arXiv 2605.26189 ↗First posted 25 May 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Quantization-aware training (QAT) with low-bit floating-point formats enables efficient LLM deployment, yet introduces subtle failure modes invisible to standard training metrics. We present a systematic study of HiF8 W8A8 QAT for OpenPangu-Embedded-1B through the lens of Delayed Tensor Scaling (DTS). Across eight controlled experiments, we identify and disentangle two orthogonal failure modes: (i)amax saturation, where delayed scale estimates silently corrupt knowledge-sensitive representations via forward-pass clipping, and (ii)catastrophic forgetting, where an aggressive learning rate overwrites pretrained commonsense knowledge independently of quantization. Neither is detectable from training loss alone. We address amax saturation with a conservative max-algorithm DTS strategy over a 64-step history window, and mitigate forgetting via a 500-step BF16 warmup followed by QAT at lr=10^{-5}. Both fixes are necessary and sufficient: our final configuration achieves 0.43% MMLU drop, 0.58% HellaSwag drop, and 0.22% ARC-Challenge drop versus a matched BF16 baseline, with a training loss APE of only 0.11% over 10,000 steps.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 13 references that are in the map. First posted on arXiv 25 May 2026.

Builds on

10 of 13 references.

References · 10

  1. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  2. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  3. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  4. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
  5. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  6. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  7. 2022FP8 Formats for Deep Learningcited by 196
  8. 2024Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsalcited by 118
  9. 2023FP8-LM: Training FP8 Large Language Modelscited by 77
  10. 2024Examining Forgetting in Continual Pre-training of Aligned Large Language Modelscited by 18

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025FP8-Flow-MoE: A Casting-Free FP8 Recipe without Double Quantization Error
  2. 2026StableQAT: Stable Quantization-Aware Training at Ultra-Low Bitwidths
  3. 2026Few Bits, One Law: Toward W2A4KV2
  4. 2026TACO: Efficient Communication Compression of Intermediate Tensors for Scalable Tensor-Parallel LLM Training
  5. 2022QFT: Post-training quantization via fast joint finetuning of all degrees of freedom
  6. 2026HiFA4: Training-Free 4-bit FlashAttention on Ascend HIF4 NPUs for LLM Inference
  7. 2026Unleashing Low-Bit Inference on Ascend NPUs: A Comprehensive Evaluation of HiFloat Formats
  8. 2026ScaleSweep: Accurate NVFP4 Post-Training Quantization of LLMs via Block Scale Initialization