# Single-Stage Huffman Encoder for ML Compression

arXiv: https://arxiv.org/abs/2601.10673 · HTML: https://mlsanity.com/paper/2601.10673/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 19 references that are in the map.
- First posted on arXiv 15 Jan 2026.

## Builds on (most cited first, 4 of 4)

- [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://mlsanity.com/paper/1909.08053/) (2019)
- [ZeRO: Memory Optimizations Toward Training Trillion Parameter Models](https://mlsanity.com/paper/1910.02054/) (2019)
- [Reducing Activation Recomputation in Large Transformer Models](https://mlsanity.com/paper/2205.05198/) (2022)
- [eXmY: A Data Type and Technique for Arbitrary Bit Precision Quantization](https://mlsanity.com/paper/2405.13938/) (2024)

## Built on by (most cited first, 2 of 2)

- [SplitZip: Ultra Fast Lossless KV Compression for Disaggregated LLM Serving](https://mlsanity.com/paper/2605.01708/) (2026)
- [Quad Length Codes for Lossless Compression of e4m3](https://mlsanity.com/paper/2602.17849/) (2026)

## Abstract (arXiv)

Training and serving Large Language Models (LLMs) require partitioning data across multiple accelerators, where collective operations are frequently bottlenecked by network bandwidth. Lossless compression using Huffman codes is an effective way to alleviate the issue, however, its three-stage design requiring on-the-fly frequency analysis, codebook generation and transmission of codebook along with data introduces computational, latency and data overheads which are prohibitive for latency-sensitive scenarios such as die-to-die communication. This paper proposes a single-stage Huffman encoder that eliminates these overheads by using fixed codebooks derived from the average probability distribution of previous data batches. Through our analysis of the Gemma 2B model, we demonstrate that tensors exhibit high statistical similarity across layers and shards. Using this approach we achieve compression within 0.5% of per-shard Huffman coding and within 1% of the ideal Shannon compressibility, enabling efficient on-the-fly compression.