# Learning Optimal Lattice Vector Quantizers for End-to-end Neural Image Compression

arXiv: https://arxiv.org/abs/2411.16119 · HTML: https://mlsanity.com/paper/2411.16119/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 50 references that are in the map.
- First posted on arXiv 25 Nov 2024.

## Builds on (most cited first, 30 of 34)

- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [High-Fidelity Generative Image Compression](https://mlsanity.com/paper/2006.09965/) (2020)
- [ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Coding](https://mlsanity.com/paper/2203.10886/) (2022)
- [CompressAI: a PyTorch library and evaluation platform for end-to-end compression research](https://mlsanity.com/paper/2011.03029/) (2020)
- [Channel-wise Autoregressive Entropy Models for Learned Image Compression](https://mlsanity.com/paper/2007.08739/) (2020)
- [Generative Adversarial Networks for Extreme Learned Image Compression](https://mlsanity.com/paper/1804.02958/) (2018)
- [Learned Image Compression with Mixed Transformer-CNN Architectures](https://mlsanity.com/paper/2303.14978/) (2023)
- [Checkerboard Context Model for Efficient Learned Image Compression](https://mlsanity.com/paper/2103.15306/) (2021)
- [Lossy Image Compression with Conditional Diffusion Models](https://mlsanity.com/paper/2209.06950/) (2022)
- [Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations](https://mlsanity.com/paper/1704.00648/) (2017)
- [Context-adaptive Entropy Model for End-to-end Optimized Image Compression](https://mlsanity.com/paper/1809.10452/) (2018)
- [Conditional Probability Models for Deep Image Compression](https://mlsanity.com/paper/1801.04260/) (2018)
- [Real-Time Adaptive Image Compression](https://mlsanity.com/paper/1705.05823/) (2017)
- [Variable Rate Deep Image Compression With a Conditional Autoencoder](https://mlsanity.com/paper/1909.04802/) (2019)
- [Towards image compression with perfect realism at ultra-low bitrates](https://mlsanity.com/paper/2310.10325/) (2023)
- [Improved Lossy Image Compression with Priming and Spatially Adaptive Bit Rates for Recurrent Networks](https://mlsanity.com/paper/1703.10114/) (2017)
- [MLIC: Multi-Reference Entropy Model for Learned Image Compression](https://mlsanity.com/paper/2211.07273/) (2022)
- [Learning Convolutional Networks for Content-weighted Image Compression](https://mlsanity.com/paper/1703.10553/) (2017)
- [Joint Global and Local Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/2112.04487/) (2021)
- [EVC: Towards Real-Time Neural Image Compression with Mask Decay](https://mlsanity.com/paper/2302.05071/) (2023)
- [Unified Multivariate Gaussian Mixture for Efficient Neural Image Compression](https://mlsanity.com/paper/2203.10897/) (2022)
- [Slimmable Compressive Autoencoders for Practical Neural Image Compression](https://mlsanity.com/paper/2103.15726/) (2021)
- [LVQAC: Lattice Vector Quantization Coupled with Spatially Adaptive Companding for Efficient Learned Image Compression](https://mlsanity.com/paper/2304.12319/) (2023)
- [Attention-guided Image Compression by Deep Reconstruction of Compressive Sensed Saliency Skeleton](https://mlsanity.com/paper/2103.15368/) (2021)
- [Multi-modality Deep Restoration of Extremely Compressed Face Videos](https://mlsanity.com/paper/2107.05548/) (2021)
- [Content Adaptive Latents and Decoder for Neural Image Compression](https://mlsanity.com/paper/2212.10132/) (2022)
- [NVTC: Nonlinear Vector Transform Coding](https://mlsanity.com/paper/2305.16025/) (2023)

## Built on by (most cited first, 8 of 8)

- [Improving 3D Gaussian Splatting Compression by Scene-Adaptive Lattice Vector Quantization](https://mlsanity.com/paper/2509.13482/) (2025)
- [Learning Grouped Lattice Vector Quantizers for Low-Bit LLM Compression](https://mlsanity.com/paper/2510.20984/) (2025)
- [Receptive Field Expanded Look-Up Tables for Vision Inference: Advancing from Low-level to High-level Tasks](https://mlsanity.com/paper/2510.10522/) (2025)
- [BADiff: Bandwidth Adaptive Diffusion Model](https://mlsanity.com/paper/2510.21366/) (2025)
- [Uni-LVC: A Unified Method for Intra- and Inter-Mode Learned Video Compression](https://mlsanity.com/paper/2603.05756/) (2026)
- [Anisotropic Pooling for LUT-realizable CNN Image Restoration](https://mlsanity.com/paper/2510.21437/) (2025)
- [HyperVQ: Enabling Hyperprior Entropy Modeling for VQ-Based Generative Image Compression](https://mlsanity.com/paper/2512.07192/) (2025)
- [Control Your View: High-Resolution Global Semantic Manipulation in Learned Image Compression](https://mlsanity.com/paper/2605.08727/) (2026)

## Abstract (arXiv)

It is customary to deploy uniform scalar quantization in the end-to-end optimized Neural image compression methods, instead of more powerful vector quantization, due to the high complexity of the latter. Lattice vector quantization (LVQ), on the other hand, presents a compelling alternative, which can exploit inter-feature dependencies more effectively while keeping computational efficiency almost the same as scalar quantization. However, traditional LVQ structures are designed/optimized for uniform source distributions, hence nonadaptive and suboptimal for real source distributions of latent code space for Neural image compression tasks. In this paper, we propose a novel learning method to overcome this weakness by designing the rate-distortion optimal lattice vector quantization (OLVQ) codebooks with respect to the sample statistics of the latent features to be compressed. By being able to better fit the LVQ structures to any given latent sample distribution, the proposed OLVQ method improves the rate-distortion performances of the existing quantization schemes in neural image compression significantly, while retaining the amenability of uniform scalar quantization.