mlsanityData to 14 Oct 2026

NF4 Isn't Information Theoretically Optimal (and that's Good)

arXiv 2306.06965 ↗First posted 12 Jun 2023cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

This note shares some simple calculations and experiments related to absmax-based blockwise quantization, as used in Dettmers et al., 2023. Their proposed NF4 data type is said to be information theoretically optimal for representing normally distributed weights. I show that this can't quite be the case, as the distribution of the values to be quantized depends on the block-size. I attempt to apply these insights to derive an improved code based on minimizing the expected L1 reconstruction error, rather than the quantile based method. This leads to improved performance for larger quantization block sizes, while both codes perform similarly at smaller block sizes.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 6 of its 20 references that are in the map. First posted on arXiv 12 Jun 2023.

Builds on

6 of 20 references.

References · 6

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  3. 2016Pointer Sentinel Mixture Modelscited by 3,096
  4. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  5. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 992
  6. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 726

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 5; 2026: 3.

11 papers.

  1. 2023LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuningcited by 92
  2. 202570% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float (DFloat11)cited by 24
  3. 2024Pushing the Limits of Large Language Model Quantization via the Linearity Theoremcited by 21
  4. 2025Scaling Laws for Floating Point Quantization Trainingcited by 8
  5. 2025Improving Block-Wise LLM Quantization by 4-bit Block-Wise Optimal Float (BOF4): Analysis and Variationscited by 5
  6. 2025any4: Learned 4-bit Numeric Representation for LLMscited by 5
  7. 2025Optimal Formats for Weight Quantisationcited by 1
  8. 2026Boosting Entropy with Bell Box Quantizationcited by 1
  9. 2024DAQ: Density-Aware Post-Training Weight-Only Quantization For LLMscited by 0
  10. 2026CodeQuant: Unified Clustering and Quantization for Enhanced Outlier Smoothing in Low-Precision Mixture-of-Expertscited by 0
  11. 2026CubicQuant: Parametric Non-Uniform Codebooks for High-Throughput LLM Inference with 1-8-Bit Weightscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Adaptive Block-Scaled Data Types
  2. 2026ScaleSweep: Accurate NVFP4 Post-Training Quantization of LLMs via Block Scale Initialization
  3. 2025Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling
  4. 20218-bit Optimizers via Block-wise Quantization
  5. 2026OSFP4: Joint Optimization of Diagonal Smoothing and Block Scales for NVFP4 Quantization
  6. 2024Accurate Block Quantization in LLMs with Outliers
  7. 2026KBBQ: A Predictive Noise Law and the Limits of Spectrum Flattening in FP4 Quantization
  8. 2026Pre-Registering the Detectable Effect: A Paired-MDE Budget for 4-bit Quantization Benchmarks, with a Pilot Audit