# Quad Length Codes for Lossless Compression of e4m3

arXiv: https://arxiv.org/abs/2602.17849 · HTML: https://mlsanity.com/paper/2602.17849/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 20 references that are in the map.
- First posted on arXiv 19 Feb 2026.

## Builds on (most cited first, 5 of 5)

- [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://mlsanity.com/paper/1909.08053/) (2019)
- [ZeRO: Memory Optimizations Toward Training Trillion Parameter Models](https://mlsanity.com/paper/1910.02054/) (2019)
- [Reducing Activation Recomputation in Large Transformer Models](https://mlsanity.com/paper/2205.05198/) (2022)
- [eXmY: A Data Type and Technique for Arbitrary Bit Precision Quantization](https://mlsanity.com/paper/2405.13938/) (2024)
- [Single-Stage Huffman Encoder for ML Compression](https://mlsanity.com/paper/2601.10673/) (2026)

## Built on by (most cited first, 1 of 1)

- [SplitZip: Ultra Fast Lossless KV Compression for Disaggregated LLM Serving](https://mlsanity.com/paper/2605.01708/) (2026)

## Abstract (arXiv)

Training and serving Large Language Models (LLMs) relies heavily on parallelization and collective operations, which are frequently bottlenecked by network bandwidth. Lossless compression using e.g., Huffman codes can alleviate the issue, however, Huffman codes suffer from slow, bit-sequential decoding and high hardware complexity due to deep tree traversals. Universal codes e.g., Exponential-Golomb codes are faster to decode but do not exploit the symbol frequency distributions. To address these limitations, this paper introduces Quad Length Codes, a hybrid approach designed to balance compression efficiency with decoding speed. The coding scheme uses 3 prefix bits to divide the 256 symbols into 8 areas. Each area has a different code length and encodes a different number of symbols. The scheme uses a Look Up Table with 256 entries, significantly simplifying the hardware implementation compared to Huffman trees. The coding scheme can be adapted for different distributions. For the e4m3 data type, the scheme achieves a compressibility of 13.9% in comparison to 15.9% achieved by Huffman codes, but it significantly speeds up the decoding and simplifies the hardware complexity.