# Advancing The Rate-Distortion-Computation Frontier For Neural Image Compression

arXiv: https://arxiv.org/abs/2311.12821 · HTML: https://mlsanity.com/paper/2311.12821/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 26 Sep 2023.

## Builds on (most cited first, 14 of 14)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Coding](https://mlsanity.com/paper/2203.10886/) (2022)
- [Channel-wise Autoregressive Entropy Models for Learned Image Compression](https://mlsanity.com/paper/2007.08739/) (2020)
- [The Devil Is in the Details: Window-based Attention for Image Compression](https://mlsanity.com/paper/2203.08450/) (2022)
- [Context-adaptive Entropy Model for End-to-end Optimized Image Compression](https://mlsanity.com/paper/1809.10452/) (2018)
- [The Efficiency Misnomer](https://mlsanity.com/paper/2110.12894/) (2021)
- [Improving Inference for Neural Image Compression](https://mlsanity.com/paper/2006.04240/) (2020)
- [Unified Multivariate Gaussian Mixture for Efficient Neural Image Compression](https://mlsanity.com/paper/2203.10897/) (2022)
- [Computationally Efficient Neural Image Compression](https://mlsanity.com/paper/1912.08771/) (2019)
- [MobileCodec: Neural Inter-frame Video Compression on Mobile Devices](https://mlsanity.com/paper/2207.08338/) (2022)

## Built on by (most cited first, 9 of 9)

- [C3: High-performance and low-complexity neural compression from a single image or video](https://mlsanity.com/paper/2312.02753/) (2023)
- [NVRC: Neural Video Representation Compression](https://mlsanity.com/paper/2409.07414/) (2024)
- [Generalized Gaussian Model for Learned Image Compression](https://mlsanity.com/paper/2411.19320/) (2024)
- [Balanced Rate-Distortion Optimization in Learned Image Compression](https://mlsanity.com/paper/2502.20161/) (2025)
- [Learned Compression for Compressed Learning](https://mlsanity.com/paper/2412.09405/) (2024)
- [Leveraging Second-Order Curvature for Efficient Learned Image Compression: Theory and Empirical Evidence](https://mlsanity.com/paper/2601.20769/) (2026)
- [Accelerating Learned Image Compression Through Modeling Neural Training Dynamics](https://mlsanity.com/paper/2505.18107/) (2025)
- [PerCoV2: Ultra-Low Bit-Rate Perceptual Image Compression via Query-Based 1D Multimodal Image Tokens](https://mlsanity.com/paper/2503.09368/) (2025)
- [LiVeAction: a Lightweight, Versatile, and Asymmetric Neural Codec Design for Real-time Operation](https://mlsanity.com/paper/2605.06628/) (2026)

## Abstract (arXiv)

The rate-distortion performance of neural image compression models has exceeded the state-of-the-art for non-learned codecs, but neural codecs are still far from widespread deployment and adoption. The largest obstacle is having efficient models that are feasible on a wide variety of consumer hardware. Comparative research and evaluation is difficult due to the lack of standard benchmarking platforms and due to variations in hardware architectures and test environments. Through our rate-distortion-computation (RDC) study we demonstrate that neither floating-point operations (FLOPs) nor runtime are sufficient on their own to accurately rank neural compression methods. We also explore the RDC frontier, which leads to a family of model architectures with the best empirical trade-off between computational requirements and RD performance. Finally, we identify a novel neural compression architecture that yields state-of-the-art RD performance with rate savings of 23.1% over BPG (7.0% over VTM and 3.0% over ELIC) without requiring significantly more FLOPs than other learning-based codecs.