# Deep Generative Models for Distribution-Preserving Lossy Compression

arXiv: https://arxiv.org/abs/1805.11057 · HTML: https://mlsanity.com/paper/1805.11057/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2018.

## Built on by (most cited first, 30 of 59)

- [Finite Scalar Quantization: VQ-VAE Made Simple](https://mlsanity.com/paper/2309.15505/) (2023)
- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [Lossy Image Compression with Conditional Diffusion Models](https://mlsanity.com/paper/2209.06950/) (2022)
- [Image-to-Image Translation: Methods and Applications](https://mlsanity.com/paper/2101.08629/) (2021)
- [Improving Statistical Fidelity for Neural Image Compression with Implicit Local Likelihood Models](https://mlsanity.com/paper/2301.11189/) (2023)
- [Lossy Compression with Gaussian Diffusion](https://mlsanity.com/paper/2206.08889/) (2022)
- [Multi-Realism Image Compression with a Conditional Generator](https://mlsanity.com/paper/2212.13824/) (2022)
- [Variable-Rate Deep Image Compression through Spatially-Adaptive Feature Transform](https://mlsanity.com/paper/2108.09551/) (2021)
- [Generative Joint Source-Channel Coding for Semantic Image Transmission](https://mlsanity.com/paper/2211.13772/) (2022)
- [Universal Rate-Distortion-Perception Representations for Lossy Compression](https://mlsanity.com/paper/2106.10311/) (2021)
- [Slimmable Compressive Autoencoders for Practical Neural Image Compression](https://mlsanity.com/paper/2103.15726/) (2021)
- [Neural Video Compression using GANs for Detail Synthesis and Propagation](https://mlsanity.com/paper/2107.12038/) (2021)
- [A Residual Diffusion Model for High Perceptual Quality Codec Augmentation](https://mlsanity.com/paper/2301.05489/) (2023)
- [Extreme Generative Image Compression by Learning Text Embedding from Diffusion Models](https://mlsanity.com/paper/2211.07793/) (2022)
- [Idempotence and Perceptual Image Compression](https://mlsanity.com/paper/2401.08920/) (2024)
- [On the advantages of stochastic encoders](https://mlsanity.com/paper/2102.09270/) (2021)
- [On Perceptual Lossy Compression: The Cost of Perceptual Reconstruction and An Optimal Training Framework](https://mlsanity.com/paper/2106.02782/) (2021)
- [DPICT: Deep Progressive Image Compression Using Trit-Planes](https://mlsanity.com/paper/2112.06334/) (2021)
- [The Rate-Distortion-Perception Tradeoff: The Role of Common Randomness](https://mlsanity.com/paper/2202.04147/) (2022)
- [PerCo (SD): Open Perceptual Compression](https://mlsanity.com/paper/2409.20255/) (2024)
- [Multi-Modality Deep Network for Extreme Learned Image Compression](https://mlsanity.com/paper/2304.13583/) (2023)
- [Once-for-All: Controllable Generative Image Compression with Dynamic Granularity Adaptation](https://mlsanity.com/paper/2406.00758/) (2024)
- [M2T: Masking Transformers Twice for Faster Decoding](https://mlsanity.com/paper/2304.07313/) (2023)
- [EGIC: Enhanced Low-Bit-Rate Generative Image Compression Guided by Semantic Segmentation](https://mlsanity.com/paper/2309.03244/) (2023)
- [Optimally Controllable Perceptual Lossy Compression](https://mlsanity.com/paper/2206.10082/) (2022)
- [ProgDTD: Progressive Learned Image Compression with Double-Tail-Drop Training](https://mlsanity.com/paper/2305.02145/) (2023)
- [A Review of Change of Variable Formulas for Generative Modeling](https://mlsanity.com/paper/2308.02652/) (2023)
- [Bottlenecks CLUB: Unifying Information-Theoretic Trade-offs Among Complexity, Leakage, and Utility](https://mlsanity.com/paper/2207.04895/) (2022)
- [PICD: Versatile Perceptual Image Compression with Diffusion Rendering](https://mlsanity.com/paper/2505.05853/) (2025)
- [PNeRV: Enhancing Spatial Consistency via Pyramidal Neural Representation for Videos](https://mlsanity.com/paper/2404.08921/) (2024)

## Abstract (arXiv)

We propose and study the problem of distribution-preserving lossy compression. Motivated by recent advances in extreme image compression which allow to maintain artifact-free reconstructions even at very low bitrates, we propose to optimize the rate-distortion tradeoff under the constraint that the reconstructed samples follow the distribution of the training data. The resulting compression system recovers both ends of the spectrum: On one hand, at zero bitrate it learns a generative model of the data, and at high enough bitrates it achieves perfect reconstruction. Furthermore, for intermediate bitrates it smoothly interpolates between learning a generative model of the training data and perfectly reconstructing the training samples. We study several methods to approximately solve the proposed optimization problem, including a novel combination of Wasserstein GAN and Wasserstein Autoencoder, and present an extensive theoretical and empirical characterization of the proposed compression systems.