mlsanityData to 14 Oct 2026

OSFP4: Joint Optimization of Diagonal Smoothing and Block Scales for NVFP4 Quantization

arXiv 2610.08231 ↗First posted 6 Oct 2026cs.AI cs.IT math.ITcited by 0

Not on the map. Why?

Abstract (arXiv)

NVFP4 is an attractive datatype for large language model (LLM) inference, offering compact storage and native tensor-core acceleration. However, preserving accuracy using NVFP4 requires careful quantization. In this work we develop a novel quantization scheme called Optimized Smoothing and Scaling for NVFP4 (OSFP4). For each linear projection it uses a diagonal smoothing matrix whose entries are optimized to minimize the squared matrix-product quantization error under NVFP4, taking into account the rounding procedure that is used (either round-to-nearest, or GPTQ-style successive interference cancellation). This requires performing joint optimization on the smoothing entries as well as the block scales, which is facilitated by analyzing a multiplicative-dither FP4 quantizer instead of the fixed deterministic one. Experiments show that OSFP4 achieves the highest average accuracy among the evaluated competitors in the corresponding quantization settings, while retaining approximately 94-97% of vendor NVFP4 prefill throughput on the measured workloads. Our code is available in https://github.com/neriahbd/OSFP4

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 21 references that are in the map. First posted on arXiv 6 Oct 2026.

Builds on

16 of 21 references.

References · 16

  1. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  2. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  3. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  4. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  5. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  6. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  7. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 131
  8. 2025Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scalingcited by 35
  9. 2025NestQuant: Nested Lattice Quantization for Matrix Products and LLMscited by 23
  10. 2025WUSH: Near-Optimal Adaptive Transforms for LLM Quantizationcited by 15
  11. 2026WaterSIC: Information-Theoretically (Near) Optimal Linear Layer Quantizationcited by 7
  12. 2026SOAR: Scale Optimization for Accurate Reconstruction in NVFP4 Quantizationcited by 4
4 more
  1. 2026High-Rate Quantized Matrix Multiplication IIcited by 4
  2. 2026Search Your Block Floating Point Scales!cited by 3
  3. 2026ScaleSweep: Accurate NVFP4 Post-Training Quantization of LLMs via Block Scale Initializationcited by 2
  4. 2026H-Scale: Hessian-Guided Scale Refinement for NVFP4 Sub-Byte LLM Inferencecited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs
  2. 2025Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization
  3. 2026FOCUS: FP4 Optimization via Coupled-Relaxation and Dual-Granularity Scaling
  4. 2026Unveiling the Potential of Quantization with MXFP4: Strategies for Quantization Error Reduction
  5. 2026MixFP4: Enhancing NVFP4 with Adaptive FP4/INT4 Block Representations
  6. 2025FP4 All the Way: Fully Quantized Training of LLMs
  7. 2026Stable FP4 Training via Transposition-Invariant Block Quantization
  8. 2026Diagnosing FP4 inference: a layer-wise and block-wise sensitivity analysis of NVFP4 and MXFP4