# Generative Latent Coding for Ultra-Low Bitrate Image and Video Compression

arXiv: https://arxiv.org/abs/2505.16177 · HTML: https://mlsanity.com/paper/2505.16177/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 77 references that are in the map.
- First posted on arXiv 22 May 2025.

## Builds on (most cited first, 30 of 48)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Pixel Recurrent Neural Networks](https://mlsanity.com/paper/1601.06759/) (2016)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [High-Fidelity Generative Image Compression](https://mlsanity.com/paper/2006.09965/) (2020)
- [DVC: An End-to-end Deep Video Compression Framework](https://mlsanity.com/paper/1812.00101/) (2018)
- [Towards Robust Blind Face Restoration with Codebook Lookup Transformer](https://mlsanity.com/paper/2206.11253/) (2022)
- [Deep Contextual Video Compression](https://mlsanity.com/paper/2109.15047/) (2021)
- [Generative Adversarial Networks for Extreme Learned Image Compression](https://mlsanity.com/paper/1804.02958/) (2018)
- [Learned Image Compression with Mixed Transformer-CNN Architectures](https://mlsanity.com/paper/2303.14978/) (2023)
- [Variational Lossy Autoencoder](https://mlsanity.com/paper/1611.02731/) (2016)
- [Neural Video Compression with Diverse Contexts](https://mlsanity.com/paper/2302.14402/) (2023)
- [FVC: A New Framework towards Deep Video Compression in Feature Space](https://mlsanity.com/paper/2105.09600/) (2021)
- [Context-adaptive Entropy Model for End-to-end Optimized Image Compression](https://mlsanity.com/paper/1809.10452/) (2018)
- [Hybrid Spatial-Temporal Entropy Modelling for Neural Video Compression](https://mlsanity.com/paper/2207.05894/) (2022)
- [Neural Video Compression with Feature Modulation](https://mlsanity.com/paper/2402.17414/) (2024)
- [Towards image compression with perfect realism at ultra-low bitrates](https://mlsanity.com/paper/2310.10325/) (2023)
- [Improving Statistical Fidelity for Neural Image Compression with Implicit Local Likelihood Models](https://mlsanity.com/paper/2301.11189/) (2023)

## Built on by (most cited first, 25 of 25)

- [Single-step Diffusion-based Video Coding with Semantic-Temporal Guidance](https://mlsanity.com/paper/2512.07480/) (2025)
- [Generative Latent Video Compression](https://mlsanity.com/paper/2510.09987/) (2025)
- [Ultra-Fast Neural Video Compression](https://mlsanity.com/paper/2606.04410/) (2026)
- [DiT-IC: Aligned Diffusion Transformer for Efficient Image Compression](https://mlsanity.com/paper/2603.13162/) (2026)
- [GVCC: Zero-Shot Video Compression via Codebook-Driven Stochastic Rectified Flow](https://mlsanity.com/paper/2603.26571/) (2026)
- [Low-Bitrate Video Compression through Semantic-Conditioned Diffusion](https://mlsanity.com/paper/2512.00408/) (2025)
- [Generative Video Compression with One-Dimensional Latent Representation](https://mlsanity.com/paper/2603.15302/) (2026)
- [DiffVC-RT: Towards Practical Real-Time Diffusion-based Perceptual Neural Video Compression](https://mlsanity.com/paper/2601.20564/) (2026)
- [ProGIC: Progressive and Lightweight Generative Image Compression with Residual Vector Quantization](https://mlsanity.com/paper/2603.02897/) (2026)
- [Enhancing Neural Video Compression of Static Scenes with Positive-Incentive Noise](https://mlsanity.com/paper/2603.06095/) (2026)
- [ProGVC: Progressive-based Generative Video Compression via Auto-Regressive Context Modeling](https://mlsanity.com/paper/2603.17546/) (2026)
- [ZeroGVC: Zero-Shot Generative Video Compression with Autoregressive Diffusion Priors](https://mlsanity.com/paper/2606.22371/) (2026)
- [CADC: Content Adaptive Diffusion-Based Generative Image Compression](https://mlsanity.com/paper/2602.21591/) (2026)
- [Compression as Adaptation: Implicit Visual Representation with Diffusion Foundation Models](https://mlsanity.com/paper/2603.07615/) (2026)
- [Uni-LVC: A Unified Method for Intra- and Inter-Mode Learned Video Compression](https://mlsanity.com/paper/2603.05756/) (2026)
- [Generative Video Compression with Adaptive Score Distillation](https://mlsanity.com/paper/2607.22772/) (2026)
- [A Causal Diffusion Model for Video Reconstruction from Ultra-Low-Bitrate Representations](https://mlsanity.com/paper/2602.13837/) (2026)
- [Geometric Transformation-Embedded Mamba for Learned Video Compression](https://mlsanity.com/paper/2603.07912/) (2026)
- [Efficient Learned Image Compression without Entropy Coding](https://mlsanity.com/paper/2605.23323/) (2026)
- [GVCCTurbo: Rate-Compute Quality Scheduling for Codebook Driven Generative Compression](https://mlsanity.com/paper/2608.03517/) (2026)
- [Generative Video Compression Based on Hierarchical Referencing](https://mlsanity.com/paper/2608.11618/) (2026)
- [DiffVC-ONE: Diffusion-based Generative Video Compression with One-Step Video Diffusion Transformer](https://mlsanity.com/paper/2608.20515/) (2026)
- [Rate-Adaptive One-Step Diffusion Compression for AIGC Images](https://mlsanity.com/paper/2609.31795/) (2026)
- [A Spatiotemporal Semantic Importance-Guided Unified Compression and Editing Framework for AI-Generated Videos](https://mlsanity.com/paper/2610.05779/) (2026)
- [Unified Generative Video Compression with Autoregressive and Bidirectional Video Diffusion](https://mlsanity.com/paper/2610.13023/) (2026)

## Abstract (arXiv)

Most existing approaches for image and video compression perform transform coding in the pixel space to reduce redundancy. However, due to the misalignment between the pixel-space distortion and human perception, such schemes often face the difficulties in achieving both high-realism and high-fidelity at ultra-low bitrate. To solve this problem, we propose Generative Latent Coding (GLC) models for image and video compression, termed GLC-image and GLC-Video. The transform coding of GLC is conducted in the latent space of a generative vector-quantized variational auto-encoder (VQ-VAE). Compared to the pixel-space, such a latent space offers greater sparsity, richer semantics and better alignment with human perception, and show its advantages in achieving high-realism and high-fidelity compression. To further enhance performance, we improve the hyper prior by introducing a spatial categorical hyper module in GLC-image and a spatio-temporal categorical hyper module in GLC-video. Additionally, the code-prediction-based loss function is proposed to enhance the semantic consistency. Experiments demonstrate that our scheme shows high visual quality at ultra-low bitrate for both image and video compression. For image compression, GLC-image achieves an impressive bitrate of less than 0.04 bpp, achieving the same FID as previous SOTA model MS-ILLM while using 45% fewer bitrate on the CLIC 2020 test set. For video compression, GLC-video achieves 65.3% bitrate saving over PLVC in terms of DISTS.