# NF4 Isn't Information Theoretically Optimal (and that's Good)

arXiv: https://arxiv.org/abs/2306.06965 · HTML: https://mlsanity.com/paper/2306.06965/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 20 references that are in the map.
- First posted on arXiv 12 Jun 2023.

## Builds on (most cited first, 6 of 6)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [The LAMBADA dataset: Word prediction requiring a broad discourse context](https://mlsanity.com/paper/1606.06031/) (2016)
- [Compressive Transformers for Long-Range Sequence Modelling](https://mlsanity.com/paper/1911.05507/) (2019)

## Built on by (most cited first, 11 of 11)

- [LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning](https://mlsanity.com/paper/2311.12023/) (2023)
- [70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float (DFloat11)](https://mlsanity.com/paper/2504.11651/) (2025)
- [Pushing the Limits of Large Language Model Quantization via the Linearity Theorem](https://mlsanity.com/paper/2411.17525/) (2024)
- [Scaling Laws for Floating Point Quantization Training](https://mlsanity.com/paper/2501.02423/) (2025)
- [Improving Block-Wise LLM Quantization by 4-bit Block-Wise Optimal Float (BOF4): Analysis and Variations](https://mlsanity.com/paper/2505.06653/) (2025)
- [any4: Learned 4-bit Numeric Representation for LLMs](https://mlsanity.com/paper/2507.04610/) (2025)
- [Optimal Formats for Weight Quantisation](https://mlsanity.com/paper/2505.12988/) (2025)
- [Boosting Entropy with Bell Box Quantization](https://mlsanity.com/paper/2603.01599/) (2026)
- [DAQ: Density-Aware Post-Training Weight-Only Quantization For LLMs](https://mlsanity.com/paper/2410.12187/) (2024)
- [CodeQuant: Unified Clustering and Quantization for Enhanced Outlier Smoothing in Low-Precision Mixture-of-Experts](https://mlsanity.com/paper/2604.10496/) (2026)
- [CubicQuant: Parametric Non-Uniform Codebooks for High-Throughput LLM Inference with 1-8-Bit Weights](https://mlsanity.com/paper/2608.06763/) (2026)

## Abstract (arXiv)

This note shares some simple calculations and experiments related to absmax-based blockwise quantization, as used in Dettmers et al., 2023. Their proposed NF4 data type is said to be information theoretically optimal for representing normally distributed weights. I show that this can't quite be the case, as the distribution of the values to be quantized depends on the block-size. I attempt to apply these insights to derive an improved code based on minimizing the expected L1 reconstruction error, rather than the quantile based method. This leads to improved performance for larger quantization block sizes, while both codes perform similarly at smaller block sizes.