# Unifying Generation and Compression: Ultra-low bitrate Image Coding Via Multi-stage Transformer

arXiv: https://arxiv.org/abs/2403.03736 · HTML: https://mlsanity.com/paper/2403.03736/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 28 references that are in the map.
- First posted on arXiv 6 Mar 2024.

## Builds on (most cited first, 17 of 17)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Holistically-Nested Edge Detection](https://mlsanity.com/paper/1504.06375/) (2015)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [High-Fidelity Generative Image Compression](https://mlsanity.com/paper/2006.09965/) (2020)
- [CompressAI: a PyTorch library and evaluation platform for end-to-end compression research](https://mlsanity.com/paper/2011.03029/) (2020)
- [Language Modeling Is Compression](https://mlsanity.com/paper/2309.10668/) (2023)
- [Generative Adversarial Networks for Extreme Learned Image Compression](https://mlsanity.com/paper/1804.02958/) (2018)
- [Checkerboard Context Model for Efficient Learned Image Compression](https://mlsanity.com/paper/2103.15306/) (2021)
- [Multi-Realism Image Compression with a Conditional Generator](https://mlsanity.com/paper/2212.13824/) (2022)
- [Fidelity-Controllable Extreme Image Compression with Generative Adversarial Networks](https://mlsanity.com/paper/2008.10314/) (2020)
- [Extreme Image Compression using Fine-tuned VQGANs](https://mlsanity.com/paper/2307.08265/) (2023)
- [High-Efficiency Lossy Image Coding Through Adaptive Neighborhood Information Aggregation](https://mlsanity.com/paper/2204.11448/) (2022)
- [Conceptual Compression via Deep Structure and Texture Synthesis](https://mlsanity.com/paper/2011.04976/) (2020)
- [Thousand to One: Semantic Prior Modeling for Conceptual Coding](https://mlsanity.com/paper/2103.07131/) (2021)

## Built on by (most cited first, 12 of 12)

- [One-Step Diffusion-Based Image Compression with Semantic Distillation](https://mlsanity.com/paper/2505.16687/) (2025)
- [DLF: Extreme Image Compression with Dual-generative Latent Fusion](https://mlsanity.com/paper/2503.01428/) (2025)
- [Once-for-All: Controllable Generative Image Compression with Dynamic Granularity Adaptation](https://mlsanity.com/paper/2406.00758/) (2024)
- [CMC-Bench: Towards a New Paradigm of Visual Signal Compression](https://mlsanity.com/paper/2406.09356/) (2024)
- [Differentiable Vector Quantization for Rate-Distortion Optimization of Generative Image Compression](https://mlsanity.com/paper/2604.10546/) (2026)
- [Next-Frame Decoding for Ultra-Low-Bitrate Image Compression with Video Diffusion Priors](https://mlsanity.com/paper/2603.15129/) (2026)
- [Why Compress What You Can Generate? When GPT-4o Generation Ushers in Image Compression Fields](https://mlsanity.com/paper/2504.21814/) (2025)
- [MRT: Learning Compact Representations with Mixed RWKV-Transformer for Extreme Image Compression](https://mlsanity.com/paper/2511.06717/) (2025)
- [Dual-Latent Collaborative Decoding for Fidelity-Perception Balanced Image Compression](https://mlsanity.com/paper/2605.14391/) (2026)
- [PerCoV2: Ultra-Low Bit-Rate Perceptual Image Compression via Query-Based 1D Multimodal Image Tokens](https://mlsanity.com/paper/2503.09368/) (2025)
- [ResARC: Residual-Aware AutoRegressive Coding for Ultra-Low Bitrate Image Compression](https://mlsanity.com/paper/2609.39451/) (2026)
- [Recasting DiT for Extreme Image Compression with Hierarchical Visual Priors](https://mlsanity.com/paper/2610.18607/) (2026)

## Abstract (arXiv)

Recent progress in generative compression technology has significantly improved the perceptual quality of compressed data. However, these advancements primarily focus on producing high-frequency details, often overlooking the ability of generative models to capture the prior distribution of image content, thus impeding further bitrate reduction in extreme compression scenarios (<0.05 bpp). Motivated by the capabilities of predictive language models for lossless compression, this paper introduces a novel Unified Image Generation-Compression (UIGC) paradigm, merging the processes of generation and compression. A key feature of the UIGC framework is the adoption of vector-quantized (VQ) image models for tokenization, alongside a multi-stage transformer designed to exploit spatial contextual information for modeling the prior distribution. As such, the dual-purpose framework effectively utilizes the learned prior for entropy estimation and assists in the regeneration of lost tokens. Extensive experiments demonstrate the superiority of the proposed UIGC framework over existing codecs in perceptual quality and human perception, particularly in ultra-low bitrate scenarios (<=0.03 bpp), pioneering a new direction in generative compression.