# Approaching Rate-Distortion Limits in Neural Compression with Lattice Transform Coding

arXiv: https://arxiv.org/abs/2403.07320 · HTML: https://mlsanity.com/paper/2403.07320/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 50 references that are in the map.
- First posted on arXiv 12 Mar 2024.

## Builds on (most cited first, 19 of 19)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Coding](https://mlsanity.com/paper/2203.10886/) (2022)
- [CompressAI: a PyTorch library and evaluation platform for end-to-end compression research](https://mlsanity.com/paper/2011.03029/) (2020)
- [Formal Limitations on the Measurement of Mutual Information](https://mlsanity.com/paper/1811.04251/) (2018)
- [Nonlinear Transform Coding](https://mlsanity.com/paper/2007.03034/) (2020)
- [An Introduction to Neural Data Compression](https://mlsanity.com/paper/2202.06533/) (2022)
- [Image Compression with Product Quantized Masked Image Modeling](https://mlsanity.com/paper/2212.07372/) (2022)
- [LVQAC: Lattice Vector Quantization Coupled with Spatially Adaptive Companding for Efficient Learned Image Compression](https://mlsanity.com/paper/2304.12319/) (2023)
- [Towards Empirical Sandwich Bounds on the Rate-Distortion Function](https://mlsanity.com/paper/2111.12166/) (2021)
- [NVTC: Nonlinear Vector Transform Coding](https://mlsanity.com/paper/2305.16025/) (2023)
- [Neural Networks Optimally Compress the Sawbridge](https://mlsanity.com/paper/2011.05065/) (2020)
- [Neural Estimation of the Rate-Distortion Function With Applications to Operational Source Coding](https://mlsanity.com/paper/2204.01612/) (2022)
- [Fundamental Limits of Two-layer Autoencoders, and Achieving Them with Gradient Methods](https://mlsanity.com/paper/2212.13468/) (2022)
- [Do Neural Networks Compress Manifolds Optimally?](https://mlsanity.com/paper/2205.08518/) (2022)
- [Principal Bit Analysis: Autoencoding with Schur-Concave Loss](https://mlsanity.com/paper/2106.02796/) (2021)

## Built on by (most cited first, 8 of 8)

- [Learning Optimal Lattice Vector Quantizers for End-to-end Neural Image Compression](https://mlsanity.com/paper/2411.16119/) (2024)
- [Improving 3D Gaussian Splatting Compression by Scene-Adaptive Lattice Vector Quantization](https://mlsanity.com/paper/2509.13482/) (2025)
- [Optimal Neural Compressors for the Rate-Distortion-Perception Tradeoff](https://mlsanity.com/paper/2503.17558/) (2025)
- [Compression Tells Intelligence: Visual Coding, Visual Token Technology, and the Unification](https://mlsanity.com/paper/2601.20742/) (2026)
- [Leveraging Second-Order Curvature for Efficient Learned Image Compression: Theory and Empirical Evidence](https://mlsanity.com/paper/2601.20769/) (2026)
- [HyperVQ: Enabling Hyperprior Entropy Modeling for VQ-Based Generative Image Compression](https://mlsanity.com/paper/2512.07192/) (2025)
- [Wavefront Parallelization for Efficient Learned Image Compression](https://mlsanity.com/paper/2607.19082/) (2026)
- [A Unified Rate-Distortion Perspective on Vector, Product, and Scalar Quantization](https://mlsanity.com/paper/2609.02107/) (2026)

## Abstract (arXiv)

Neural compression has brought tremendous progress in designing lossy compressors with good rate-distortion (RD) performance at low complexity. Thus far, neural compression design involves transforming the source to a latent vector, which is then rounded to integers and entropy coded. While this approach has been shown to be optimal on a few specific sources, we show that it can be highly sub-optimal on synthetic sources whose intrinsic dimensionality is greater than one. With integer rounding in the latent space, the quantization regions induced by neural transformations, remain square-like and fail to match those of optimal vector quantization. We demonstrate that this phenomenon is due to the choice of scalar quantization in the latent space, and not the transform design. By employing lattice quantization instead, we propose Lattice Transform Coding (LTC) and show that it approximately recovers optimal vector quantization at reasonable complexity. On real-world sources, LTC improves upon standard neural compressors. LTC also provides a framework that can integrate structurally (near) optimal information-theoretic designs into lossy compression; examples include block coding, which yields coding gain over optimal one-shot coding and approaches the asymptotically-achievable rate-distortion function, as well as nested lattice quantization for low complexity fixed-rate coding.