# Lossy Image Compression with Quantized Hierarchical VAEs

arXiv: https://arxiv.org/abs/2208.13056 · HTML: https://mlsanity.com/paper/2208.13056/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 58 references that are in the map.
- First posted on arXiv 27 Aug 2022.

## Builds on (most cited first, 30 of 43)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [NVAE: A Deep Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2007.03898/) (2020)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [Ladder Variational Autoencoders](https://mlsanity.com/paper/1602.02282/) (2016)
- [Lossy Image Compression with Compressive Autoencoders](https://mlsanity.com/paper/1703.00395/) (2017)
- [CompressAI: a PyTorch library and evaluation platform for end-to-end compression research](https://mlsanity.com/paper/2011.03029/) (2020)
- [Channel-wise Autoregressive Entropy Models for Learned Image Compression](https://mlsanity.com/paper/2007.08739/) (2020)
- [Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images](https://mlsanity.com/paper/2011.10650/) (2020)
- [Checkerboard Context Model for Efficient Learned Image Compression](https://mlsanity.com/paper/2103.15306/) (2021)
- [The Devil Is in the Details: Window-based Attention for Image Compression](https://mlsanity.com/paper/2203.08450/) (2022)
- [Nonlinear Transform Coding](https://mlsanity.com/paper/2007.03034/) (2020)
- [Discrete Variational Autoencoders](https://mlsanity.com/paper/1609.02200/) (2016)
- [Enhanced Invertible Encoding for Learned Image Compression](https://mlsanity.com/paper/2108.03690/) (2021)
- [Non-local Attention Optimized Deep Image Compression](https://mlsanity.com/paper/1904.09757/) (2019)
- [Practical Lossless Compression with Latent Variables using Bits Back Coding](https://mlsanity.com/paper/1901.04866/) (2019)
- [End-to-end optimization of nonlinear transform codes for perceptual quality](https://mlsanity.com/paper/1607.05006/) (2016)
- [Universally Quantized Neural Compression](https://mlsanity.com/paper/2006.09952/) (2020)
- [Transformer-based Image Compression](https://mlsanity.com/paper/2111.06707/) (2021)

## Built on by (most cited first, 30 of 35)

- [MLIC++: Linear Complexity Multi-Reference Entropy Modeling for Learned Image Compression](https://mlsanity.com/paper/2307.15421/) (2023)
- [QARV: Quantization-Aware ResNet VAE for Lossy Image Compression](https://mlsanity.com/paper/2302.08899/) (2023)
- [MambaVC: Learned Visual Compression with Selective State Spaces](https://mlsanity.com/paper/2405.15413/) (2024)
- [Diffusion-Driven Semantic Communication for Generative Models with Bandwidth Constraints](https://mlsanity.com/paper/2407.18468/) (2024)
- [All-in-One Image Coding for Joint Human-Machine Vision with Multi-Path Aggregation](https://mlsanity.com/paper/2409.19660/) (2024)
- [Dynamic Low-Rank Instance Adaptation for Universal Neural Image Compression](https://mlsanity.com/paper/2308.07733/) (2023)
- [Progressive Compression with Universally Quantized Diffusion Models](https://mlsanity.com/paper/2412.10935/) (2024)
- [GIViC: Generative Implicit Video Compression](https://mlsanity.com/paper/2503.19604/) (2025)
- [Generalized Nested Latent Variable Models for Lossy Coding applied to Wind Turbine Scenarios](https://mlsanity.com/paper/2406.06165/) (2024)
- [Another Way to the Top: Exploit Contextual Clustering in Learned Image Coding](https://mlsanity.com/paper/2401.11615/) (2024)
- [STanH : Parametric Quantization for Variable Rate Learned Image Compression](https://mlsanity.com/paper/2410.00557/) (2024)
- [JPEG AI Image Compression Visual Artifacts: Detection Methods and Dataset](https://mlsanity.com/paper/2411.06810/) (2024)
- [Flexible Mixed Precision Quantization for Learned Image Compression](https://mlsanity.com/paper/2506.01221/) (2025)
- [Exploring adversarial robustness of JPEG AI: methodology, comparison and new methods](https://mlsanity.com/paper/2411.11795/) (2024)
- [Learned Image Transmission with Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2408.16340/) (2024)
- [High-Efficiency Neural Video Compression via Hierarchical Predictive Learning](https://mlsanity.com/paper/2410.02598/) (2024)
- [Lightweight Embedded FPGA Deployment of Learned Image Compression with Knowledge Distillation and Hybrid Quantization](https://mlsanity.com/paper/2503.04832/) (2025)
- [CodecSplat: Ultra-Compact Latent Coding for Feed-Forward 3D Gaussian Splatting](https://mlsanity.com/paper/2605.25563/) (2026)
- [Generative AI for Rapid Diffusion MRI with Improved Image Quality, Reliability and Generalizability](https://mlsanity.com/paper/2303.05686/) (2023)
- [CPIPS: Learning to Preserve Perceptual Distances in End-to-End Image Compression](https://mlsanity.com/paper/2310.00559/) (2023)
- [Effortless Cross-Platform Video Codec: A Codebook-Based Method](https://mlsanity.com/paper/2310.10292/) (2023)
- [Learned Image Compression with Text Quality Enhancement](https://mlsanity.com/paper/2402.08643/) (2024)
- [Gone With the Bits: Revealing Racial Bias in Low-Rate Neural Compression for Facial Images](https://mlsanity.com/paper/2505.02949/) (2025)
- [Structured Pruning and Quantization for Learned Image Compression](https://mlsanity.com/paper/2506.01229/) (2025)
- [NIC-RobustBench: A Comprehensive Open-Source Toolkit for Neural Image Compression and Robustness Analysis](https://mlsanity.com/paper/2506.19051/) (2025)
- [Minimal High-Resolution Patches Are Sufficient for Whole Slide Image Representation via Cascaded Dual-Scale Reconstruction](https://mlsanity.com/paper/2508.01641/) (2025)
- [CIC: Circular Image Compression](https://mlsanity.com/paper/2407.15870/) (2024)
- [Beta-Sigma VAE: Separating beta and decoder variance in Gaussian variational autoencoder](https://mlsanity.com/paper/2409.09361/) (2024)
- [Edge-based Denoising Image Compression](https://mlsanity.com/paper/2409.10978/) (2024)
- [Interleaved Block-based Learned Image Compression with Feature Enhancement and Quantization Error Compensation](https://mlsanity.com/paper/2502.15188/) (2025)

## Abstract (arXiv)

Recent research has shown a strong theoretical connection between variational autoencoders (VAEs) and the rate-distortion theory. Motivated by this, we consider the problem of lossy image compression from the perspective of generative modeling. Starting with ResNet VAEs, which are originally designed for data (image) distribution modeling, we redesign their latent variable model using a quantization-aware posterior and prior, enabling easy quantization and entropy coding at test time. Along with improved neural network architecture, we present a powerful and efficient model that outperforms previous methods on natural image lossy compression. Our model compresses images in a coarse-to-fine fashion and supports parallel encoding and decoding, leading to fast execution on GPUs. Code is available at https://github.com/duanzhiihao/lossy-vae.