# Towards image compression with perfect realism at ultra-low bitrates

arXiv: https://arxiv.org/abs/2310.10325 · HTML: https://mlsanity.com/paper/2310.10325/

## Key facts

- Cited by 103 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 55 references that are in the map.
- First posted on arXiv 16 Oct 2023.

## Builds on (most cited first, 30 of 42)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Vector-quantized Image Modeling with Improved VQGAN](https://mlsanity.com/paper/2110.04627/) (2021)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [The Perception-Distortion Tradeoff](https://mlsanity.com/paper/1711.06077/) (2017)
- [Vision Transformer Adapter for Dense Predictions](https://mlsanity.com/paper/2205.08534/) (2022)
- [High-Fidelity Generative Image Compression](https://mlsanity.com/paper/2006.09965/) (2020)
- [Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery](https://mlsanity.com/paper/2302.03668/) (2023)
- [ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Coding](https://mlsanity.com/paper/2203.10886/) (2022)
- [Channel-wise Autoregressive Entropy Models for Learned Image Compression](https://mlsanity.com/paper/2007.08739/) (2020)
- [Generative Adversarial Networks for Extreme Learned Image Compression](https://mlsanity.com/paper/1804.02958/) (2018)

## Built on by (most cited first, 30 of 103)

- [Towards Extreme Image Compression with Latent Feature Guidance and Diffusion Prior](https://mlsanity.com/paper/2404.18820/) (2024)
- [Lossy Image Compression with Foundation Diffusion Models](https://mlsanity.com/paper/2404.08580/) (2024)
- [Consistency-diversity-realism Pareto fronts of conditional image generative models](https://mlsanity.com/paper/2406.10429/) (2024)
- [StableCodec: Taming One-Step Diffusion for Extreme Image Compression](https://mlsanity.com/paper/2506.21977/) (2025)
- [Compressed Image Generation with Denoising Diffusion Codebook Models](https://mlsanity.com/paper/2502.01189/) (2025)
- [One-Step Diffusion-Based Image Compression with Semantic Distillation](https://mlsanity.com/paper/2505.16687/) (2025)
- [Lossy Compression with Pretrained Diffusion Models](https://mlsanity.com/paper/2501.09815/) (2025)
- [Generative Latent Coding for Ultra-Low Bitrate Image and Video Compression](https://mlsanity.com/paper/2505.16177/) (2025)
- [Neural Image Compression with Text-guided Encoding for both Pixel-level and Perceptual Fidelity](https://mlsanity.com/paper/2403.02944/) (2024)
- [RDEIC: Accelerating Diffusion-Based Extreme Image Compression with Relay Residual Diffusion](https://mlsanity.com/paper/2410.02640/) (2024)
- [Generative Latent Coding for Ultra-Low Bitrate Image Compression](https://mlsanity.com/paper/2512.20194/) (2025)
- [DLF: Extreme Image Compression with Dual-generative Latent Fusion](https://mlsanity.com/paper/2503.01428/) (2025)
- [Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusion](https://mlsanity.com/paper/2505.08281/) (2025)
- [PerCo (SD): Open Perceptual Compression](https://mlsanity.com/paper/2409.20255/) (2024)
- [Semantics-Guided Diffusion for Deep Joint Source-Channel Coding in Wireless Image Transmission](https://mlsanity.com/paper/2501.01138/) (2025)
- [Single-step Diffusion-based Video Coding with Semantic-Temporal Guidance](https://mlsanity.com/paper/2512.07480/) (2025)
- [On Improved Conditioning Mechanisms and Pre-training Strategies for Diffusion Models](https://mlsanity.com/paper/2411.03177/) (2024)
- [Diffusion-Aided Joint Source Channel Coding For High Realism Wireless Image Transmission](https://mlsanity.com/paper/2404.17736/) (2024)
- [PICD: Versatile Perceptual Image Compression with Diffusion Rendering](https://mlsanity.com/paper/2505.05853/) (2025)
- [Generative Latent Video Compression](https://mlsanity.com/paper/2510.09987/) (2025)
- [What makes an image realistic?](https://mlsanity.com/paper/2403.04493/) (2024)
- [M3-CVC: Controllable Video Compression with Multimodal Generative Models](https://mlsanity.com/paper/2411.15798/) (2024)
- [Progressive Compression with Universally Quantized Diffusion Models](https://mlsanity.com/paper/2412.10935/) (2024)
- [CoD: A Diffusion Foundation Model for Image Compression](https://mlsanity.com/paper/2511.18706/) (2025)
- [Generative Video Semantic Communication via Multimodal Semantic Fusion with Large Model](https://mlsanity.com/paper/2502.13838/) (2025)
- [PSC: Posterior Sampling-Based Compression](https://mlsanity.com/paper/2407.09896/) (2024)
- [Learning Optimal Lattice Vector Quantizers for End-to-end Neural Image Compression](https://mlsanity.com/paper/2411.16119/) (2024)
- [Qinco2: Vector Compression and Search with Improved Implicit Neural Codebooks](https://mlsanity.com/paper/2501.03078/) (2025)
- [Bridging the Gap between Gaussian Diffusion Models and Universal Quantization for Image Compression](https://mlsanity.com/paper/2504.02579/) (2025)
- [Turbo-DDCM: Fast and Flexible Zero-Shot Diffusion-Based Image Compression](https://mlsanity.com/paper/2511.06424/) (2025)

## Abstract (arXiv)

Image codecs are typically optimized to trade-off bitrate vs distortion metrics. At low bitrates, this leads to compression artefacts which are easily perceptible, even when training with perceptual or adversarial losses. To improve image quality and remove dependency on the bitrate, we propose to decode with iterative diffusion models. We condition the decoding process on a vector-quantized image representation, as well as a global image description to provide additional context. We dub our model PerCo for 'perceptual compression', and compare it to state-of-the-art codecs at rates from 0.1 down to 0.003 bits per pixel. The latter rate is more than an order of magnitude smaller than those considered in most prior work, compressing a 512x768 Kodak image with less than 153 bytes. Despite this ultra-low bitrate, our approach maintains the ability to reconstruct realistic images. We find that our model leads to reconstructions with state-of-the-art visual quality as measured by FID and KID. As predicted by rate-distortion-perception theory, visual quality is less dependent on the bitrate than previous methods.