mlsanityData to 7 Oct 2026

Adaptive Block-Scaled Data Types

arXiv 2603.28765 ↗First posted 30 Mar 2026cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

NVFP4 has grown increasingly popular as a 4-bit format for quantizing large language models due to its hardware support and its ability to retain useful information with relatively few bits per parameter. However, the format is not without limitations: recent work has shown that NVFP4 suffers from its error distribution, resulting in large amounts of quantization error on near-maximal values in each group of 16 values. In this work, we leverage this insight to design new Adaptive Block-Scaled Data Types that can adapt to the distribution of their input values. For four-bit quantization, our proposed IF4 (Int/Float 4) data type selects between FP4 and INT4 representations for each group of 16 values, which are then scaled by an E4M3 scale factor as is done with NVFP4. The selected data type is denoted using the scale factor's sign bit, which is currently unused in NVFP4, and we apply the same insight to design formats for other bit-widths, including IF3 and IF6. When used to quantize language models, we find that IF4 outperforms existing 4-bit block-scaled formats, achieving lower loss during quantized training and achieving higher accuracy on many tasks in post-training quantization. We additionally design and evaluate an IF4 Multiply-Accumulate (MAC) unit to demonstrate that IF4 can be implemented efficiently in next-generation hardware accelerators. Our code is available at https://github.com/mit-han-lab/fouroversix.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 28 of its 43 references that are in the map. First posted on arXiv 30 Mar 2026.

Builds on

28 of 43 references.

References · 28

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  3. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  6. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  7. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  8. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  9. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  10. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
  11. 2024SpinQuant: LLM quantization with learned rotationscited by 446
  12. 2017Efficient Processing of Deep Neural Networks: A Tutorial and Surveycited by 427
16 more
  1. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 354
  2. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 247
  3. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 200
  4. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 116
  5. 2021VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inferencecited by 59
  6. 2022ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantizationcited by 42
  7. 2025Quartet: Native FP4 Training Can Be Optimal for Large Language Modelscited by 37
  8. 2025FP4 All the Way: Fully Quantized Training of LLMscited by 37
  9. 2025Mind the Memory Gap: Unveiling GPU Bottlenecks in Large-Batch LLM Inferencecited by 32
  10. 2025Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scalingcited by 29
  11. 2024Learning from Students: Applying t-Distributions to Explore Accurate and Efficient Formats for LLMscited by 18
  12. 2025INT v.s. FP: A Comprehensive Study of Fine-Grained Low-bit Quantization Formatscited by 16
  13. 2025ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inferencecited by 16
  14. 2026Quartet II: Accurate LLM Pre-Training in NVFP4 by Improved Unbiased Gradient Estimationcited by 16
  15. 2025TetraJet-v2: Accurate NVFP4 Training for Large Language Models with Oscillation Suppression and Outlier Controlcited by 13
  16. 2025WUSH: Near-Optimal Adaptive Transforms for LLM Quantizationcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Grid Games: The Power of Multiple Grids for Quantizing Large Language Modelscited by 2
  2. 2026AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantizationcited by 1
  3. 2026CubicQuant: Parametric Non-Uniform Codebooks for High-Throughput LLM Inference with 1-8-Bit Weightscited by 0