# Integer Discrete Flows and Lossless Compression

arXiv: https://arxiv.org/abs/1905.07376 · HTML: https://mlsanity.com/paper/1905.07376/

## Key facts

- Cited by 83 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 May 2019.

## Built on by (most cited first, 30 of 83)

- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions](https://mlsanity.com/paper/2102.05379/) (2021)
- [Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning](https://mlsanity.com/paper/2208.04202/) (2022)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [Language Modeling Is Compression](https://mlsanity.com/paper/2309.10668/) (2023)
- [Autoregressive Diffusion Models](https://mlsanity.com/paper/2110.02037/) (2021)
- [An Introduction to Neural Data Compression](https://mlsanity.com/paper/2202.06533/) (2022)
- [Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codes](https://mlsanity.com/paper/2111.12701/) (2021)
- [Soft then Hard: Rethinking the Quantization in Neural Image Compression](https://mlsanity.com/paper/2104.05168/) (2021)
- [Learning Partial Equivariances from Data](https://mlsanity.com/paper/2110.10211/) (2021)
- [Variational Causal Networks: Approximate Bayesian Inference over Causal Structures](https://mlsanity.com/paper/2106.07635/) (2021)
- [Fair Normalizing Flows](https://mlsanity.com/paper/2106.05937/) (2021)
- [On the Out-of-distribution Generalization of Probabilistic Image Modelling](https://mlsanity.com/paper/2109.02639/) (2021)
- [iVPF: Numerical Invertible Volume Preserving Flow for Efficient Lossless Compression](https://mlsanity.com/paper/2103.16211/) (2021)
- [iFlow: Numerically Invertible Flows for Efficient Lossless Compression via a Uniform Coder](https://mlsanity.com/paper/2111.00965/) (2021)
- [Conditional Invertible Neural Networks for Diverse Image-to-Image Translation](https://mlsanity.com/paper/2105.02104/) (2021)
- [Deep Lossy Plus Residual Coding for Lossless and Near-lossless Image Compression](https://mlsanity.com/paper/2209.04847/) (2022)
- [What is the Relationship between Tensor Factorizations and Circuits (and How Can We Exploit it)?](https://mlsanity.com/paper/2409.07953/) (2024)
- [ANFIC: Image Compression Using Augmented Normalizing Flows](https://mlsanity.com/paper/2107.08470/) (2021)
- [Lossless Compression with Probabilistic Circuits](https://mlsanity.com/paper/2111.11632/) (2021)
- [Improving Lossless Compression Rates via Monte Carlo Bits-Back Coding](https://mlsanity.com/paper/2102.11086/) (2021)
- [Deep Generative Models for Detector Signature Simulation: A Taxonomic Review](https://mlsanity.com/paper/2312.09597/) (2023)
- [Sparse Probabilistic Circuits via Pruning and Growing](https://mlsanity.com/paper/2211.12551/) (2022)
- [Learning Scalable ℓ_∞-constrained Near-lossless Image Compression via Joint Lossy Image and Residual Compression](https://mlsanity.com/paper/2103.17015/) (2021)
- [Personalized Public Policy Analysis in Social Sciences using Causal-Graphical Normalizing Flows](https://mlsanity.com/paper/2202.03281/) (2022)
- [Towards Empirical Sandwich Bounds on the Rate-Distortion Function](https://mlsanity.com/paper/2111.12166/) (2021)
- [Compression via Pre-trained Transformers: A Study on Byte-Level Multimodal Data](https://mlsanity.com/paper/2410.05078/) (2024)
- [LC-FDNet: Learned Lossless Image Compression with Frequency Decomposition Network](https://mlsanity.com/paper/2112.06417/) (2021)
- [aiWave: Volumetric Image Compression with 3-D Trained Affine Wavelet-like Transform](https://mlsanity.com/paper/2203.05822/) (2022)
- [Semi-Discrete Normalizing Flows through Differentiable Tessellation](https://mlsanity.com/paper/2203.06832/) (2022)

## Abstract (arXiv)

Lossless compression methods shorten the expected representation size of data without loss of information, using a statistical model. Flow-based models are attractive in this setting because they admit exact likelihood optimization, which is equivalent to minimizing the expected number of bits per message. However, conventional flows assume continuous data, which may lead to reconstruction errors when quantized for compression. For that reason, we introduce a flow-based generative model for ordinal discrete data called Integer Discrete Flow (IDF): a bijective integer map that can learn rich transformations on high-dimensional data. As building blocks for IDFs, we introduce a flexible transformation layer called integer discrete coupling. Our experiments show that IDFs are competitive with other flow-based generative models. Furthermore, we demonstrate that IDF based compression achieves state-of-the-art lossless compression rates on CIFAR10, ImageNet32, and ImageNet64. To the best of our knowledge, this is the first lossless compression method that uses invertible neural networks.