# Understanding Entropy Coding With Asymmetric Numeral Systems (ANS): a Statistician's Perspective

arXiv: https://arxiv.org/abs/2201.01741 · HTML: https://mlsanity.com/paper/2201.01741/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 27 references that are in the map.
- First posted on arXiv 5 Jan 2022.

## Builds on (most cited first, 10 of 10)

- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Full Resolution Image Compression with Recurrent Neural Networks](https://mlsanity.com/paper/1608.05148/) (2016)
- [Practical Lossless Compression with Latent Variables using Bits Back Coding](https://mlsanity.com/paper/1901.04866/) (2019)
- [Improving Inference for Neural Image Compression](https://mlsanity.com/paper/2006.04240/) (2020)
- [Bit-Swap: Recursive Bits-Back Coding for Lossless Compression with Hierarchical Latent Variables](https://mlsanity.com/paper/1905.06845/) (2019)
- [Dynamic Word Embeddings](https://mlsanity.com/paper/1702.08359/) (2017)
- [Hierarchical Autoregressive Modeling for Neural Video Compression](https://mlsanity.com/paper/2010.10258/) (2020)
- [Insights from Generative Modeling for Neural Video Compression](https://mlsanity.com/paper/2107.13136/) (2021)

## Built on by (most cited first, 12 of 12)

- [An Introduction to Neural Data Compression](https://mlsanity.com/paper/2202.06533/) (2022)
- [Boosting Neural Representations for Videos with a Conditional Decoder](https://mlsanity.com/paper/2402.18152/) (2024)
- [GSCodec Studio: A Modular Framework for Gaussian Splat Compression](https://mlsanity.com/paper/2506.01822/) (2025)
- [PSC: Posterior Sampling-Based Compression](https://mlsanity.com/paper/2407.09896/) (2024)
- [Lossy and Lossless (L2) Post-training Model Size Compression](https://mlsanity.com/paper/2308.04269/) (2023)
- [Rejection-Sampled Universal Quantization for Smaller Quantization Errors](https://mlsanity.com/paper/2402.03030/) (2024)
- [An Efficient Implicit Neural Representation Image Codec Based on Mixed Autoregressive Model for Low-Complexity Decoding](https://mlsanity.com/paper/2401.12587/) (2024)
- [INR-MDSQC: Implicit Neural Representation Multiple Description Scalar Quantization for robust image Coding](https://mlsanity.com/paper/2306.13919/) (2023)
- [Transforming harmonic coefficients for 3D splat compression](https://mlsanity.com/paper/2609.15735/) (2026)
- [Implicit Neural Multiple Description for DNA-based data storage](https://mlsanity.com/paper/2309.06956/) (2023)
- [Reducing Storage of Pretrained Neural Networks by Rate-Constrained Quantization and Entropy Coding](https://mlsanity.com/paper/2505.18758/) (2025)
- [PaaF: Raising the perceived quality of INR-Based Image Compression](https://mlsanity.com/paper/2606.21655/) (2026)

## Abstract (arXiv)

Entropy coding is the backbone data compression. Novel machine-learning based compression methods often use a new entropy coder called Asymmetric Numeral Systems (ANS) [Duda et al., 2015], which provides very close to optimal bitrates and simplifies [Townsend et al., 2019] advanced compression techniques such as bits-back coding. However, researchers with a background in machine learning often struggle to understand how ANS works, which prevents them from exploiting its full versatility. This paper is meant as an educational resource to make ANS more approachable by presenting it from a new perspective of latent variable models and the so-called bits-back trick. We guide the reader step by step to a complete implementation of ANS in the Python programming language, which we then generalize for more advanced use cases. We also present and empirically evaluate an open-source library of various entropy coders designed for both research and production use. Related teaching videos and problem sets are available online.