# Unified Multivariate Gaussian Mixture for Efficient Neural Image Compression

arXiv: https://arxiv.org/abs/2203.10897 · HTML: https://mlsanity.com/paper/2203.10897/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 47 references that are in the map.
- First posted on arXiv 21 Mar 2022.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Large Batch Optimization for Deep Learning: Training BERT in 76 minutes](https://mlsanity.com/paper/1904.00962/) (2019)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications](https://mlsanity.com/paper/1701.05517/) (2017)
- [Checkerboard Context Model for Efficient Learned Image Compression](https://mlsanity.com/paper/2103.15306/) (2021)
- [Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations](https://mlsanity.com/paper/1704.00648/) (2017)
- [Context-adaptive Entropy Model for End-to-end Optimized Image Compression](https://mlsanity.com/paper/1809.10452/) (2018)
- [Conditional Probability Models for Deep Image Compression](https://mlsanity.com/paper/1801.04260/) (2018)
- [Learning Accurate Entropy Model with Global Reference for Image Compression](https://mlsanity.com/paper/2010.08321/) (2020)
- [Jpeg Image Compression Using Discrete Cosine Transform - A Survey](https://mlsanity.com/paper/1405.6147/) (2014)
- [Beyond Product Quantization: Deep Progressive Quantization for Image Retrieval](https://mlsanity.com/paper/1906.06698/) (2019)
- [A Survey: Various Techniques of Image Compression](https://mlsanity.com/paper/1311.6877/) (2013)

## Built on by (most cited first, 30 of 43)

- [MLIC: Multi-Reference Entropy Model for Learned Image Compression](https://mlsanity.com/paper/2211.07273/) (2022)
- [MLIC++: Linear Complexity Multi-Reference Entropy Modeling for Learned Image Compression](https://mlsanity.com/paper/2307.15421/) (2023)
- [Image-GS: Content-Adaptive Image Representation via 2D Gaussians](https://mlsanity.com/paper/2407.01866/) (2024)
- [Compressed Image Generation with Denoising Diffusion Codebook Models](https://mlsanity.com/paper/2502.01189/) (2025)
- [LVQAC: Lattice Vector Quantization Coupled with Spatially Adaptive Companding for Efficient Learned Image Compression](https://mlsanity.com/paper/2304.12319/) (2023)
- [Single Image Depth Prediction Made Better: A Multivariate Gaussian Take](https://mlsanity.com/paper/2303.18164/) (2023)
- [NVTC: Nonlinear Vector Transform Coding](https://mlsanity.com/paper/2305.16025/) (2023)
- [Dynamic Low-Rank Instance Adaptation for Universal Neural Image Compression](https://mlsanity.com/paper/2308.07733/) (2023)
- [Advancing The Rate-Distortion-Computation Frontier For Neural Image Compression](https://mlsanity.com/paper/2311.12821/) (2023)
- [Learning Optimal Lattice Vector Quantizers for End-to-end Neural Image Compression](https://mlsanity.com/paper/2411.16119/) (2024)
- [LLIC: Large Receptive Field Transform Coding with Adaptive Weights for Learned Image Compression](https://mlsanity.com/paper/2304.09571/) (2023)
- [Exploring Resolution Fields for Scalable Image Compression with Uncertainty Guidance](https://mlsanity.com/paper/2306.08941/) (2023)
- [Multi-Sample Training for Neural Image Compression](https://mlsanity.com/paper/2209.13834/) (2022)
- [VQ-Flow: Taming Normalizing Flows for Multi-Class Anomaly Detection via Hierarchical Vector Quantization](https://mlsanity.com/paper/2409.00942/) (2024)
- [Is JPEG AI going to change image forensics?](https://mlsanity.com/paper/2412.03261/) (2024)
- [An Efficient Adaptive Compression Method for Human Perception and Machine Vision Tasks](https://mlsanity.com/paper/2501.04329/) (2025)
- [Balanced Rate-Distortion Optimization in Learned Image Compression](https://mlsanity.com/paper/2502.20161/) (2025)
- [Differentiable Vector Quantization for Rate-Distortion Optimization of Generative Image Compression](https://mlsanity.com/paper/2604.10546/) (2026)
- [Neural Image Compression: Generalization, Robustness, and Spectral Biases](https://mlsanity.com/paper/2307.08657/) (2023)
- [Make Lossy Compression Meaningful for Low-Light Images](https://mlsanity.com/paper/2305.15030/) (2023)
- [You Can Mask More For Extremely Low-Bitrate Image Compression](https://mlsanity.com/paper/2306.15561/) (2023)
- [RFD-ECNet: Extreme Underwater Image Compression with Reference to Feature Dictionar](https://mlsanity.com/paper/2308.08721/) (2023)
- [ProGIC: Progressive and Lightweight Generative Image Compression with Residual Vector Quantization](https://mlsanity.com/paper/2603.02897/) (2026)
- [A Lower Bound of Hash Codes' Performance](https://mlsanity.com/paper/2210.05899/) (2022)
- [Learned Focused Plenoptic Image Compression with Microimage Preprocessing and Global Attention](https://mlsanity.com/paper/2305.00489/) (2023)
- [Exploring Effective Mask Sampling Modeling for Neural Image Compression](https://mlsanity.com/paper/2306.05704/) (2023)
- [ConvNeXt-ChARM: ConvNeXt-based Transform for Efficient Neural Image Compression](https://mlsanity.com/paper/2307.06342/) (2023)
- [Leveraging Second-Order Curvature for Efficient Learned Image Compression: Theory and Empirical Evidence](https://mlsanity.com/paper/2601.20769/) (2026)
- [AICT: An Adaptive Image Compression Transformer](https://mlsanity.com/paper/2307.06091/) (2023)
- [Effortless Cross-Platform Video Codec: A Codebook-Based Method](https://mlsanity.com/paper/2310.10292/) (2023)

## Abstract (arXiv)

Modeling latent variables with priors and hyperpriors is an essential problem in variational image compression. Formally, trade-off between rate and distortion is handled well if priors and hyperpriors precisely describe latent variables. Current practices only adopt univariate priors and process each variable individually. However, we find inter-correlations and intra-correlations exist when observing latent variables in a vectorized perspective. These findings reveal visual redundancies to improve rate-distortion performance and parallel processing ability to speed up compression. This encourages us to propose a novel vectorized prior. Specifically, a multivariate Gaussian mixture is proposed with means and covariances to be estimated. Then, a novel probabilistic vector quantization is utilized to effectively approximate means, and remaining covariances are further induced to a unified mixture and solved by cascaded estimation without context models involved. Furthermore, codebooks involved in quantization are extended to multi-codebooks for complexity reduction, which formulates an efficient compression procedure. Extensive experiments on benchmark datasets against state-of-the-art indicate our model has better rate-distortion performance and an impressive 3.18× compression speed up, giving us the ability to perform real-time, high-quality variational image compression in practice. Our source code is publicly available at urlhttps://github.com/xiaosu-zhu/McQuic.