mlsanityData to 14 Oct 2026

Exploring the Rate-Distortion-Complexity Optimization in Neural Image Compression

arXiv 2305.07678 ↗First posted 12 May 2023eess.IV cs.IT cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Despite a short history, neural image codecs have been shown to surpass classical image codecs in terms of rate-distortion performance. However, most of them suffer from significantly longer decoding times, which hinders the practical applications of neural image codecs. This issue is especially pronounced when employing an effective yet time-consuming autoregressive context model since it would increase entropy decoding time by orders of magnitude. In this paper, unlike most previous works that pursue optimal RD performance while temporally overlooking the coding complexity, we make a systematical investigation on the rate-distortion-complexity (RDC) optimization in neural image compression. By quantifying the decoding complexity as a factor in the optimization goal, we are now able to precisely control the RDC trade-off and then demonstrate how the rate-distortion performance of neural image codecs could adapt to various complexity demands. Going beyond the investigation of RDC optimization, a variable-complexity neural codec is designed to leverage the spatial dependencies adaptively according to industrial demands, which supports fine-grained complexity adjustment by balancing the RDC tradeoff. By implementing this scheme in a powerful base model, we demonstrate the feasibility and flexibility of RDC optimization for neural image codecs.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 31 of its 59 references that are in the map. First posted on arXiv 12 May 2023.

Builds on

31 of 59 references.

References · 31

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  4. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  5. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  6. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
  7. 2018Variational image compression with a scale hyperpriorcited by 935
  8. 2016Conditional Image Generation with PixelCNN Decoderscited by 862
  9. 2016Pixel Recurrent Neural Networkscited by 816
  10. 2016End-to-end Optimized Image Compressioncited by 696
  11. 2018Joint Autoregressive and Hierarchical Priors for Learned Image Compressioncited by 664
  12. 2020Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modulescited by 590
19 more
  1. 2022ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Codingcited by 301
  2. 2020CompressAI: a PyTorch library and evaluation platform for end-to-end compression researchcited by 276
  3. 2020Channel-wise Autoregressive Entropy Models for Learned Image Compressioncited by 264
  4. 2021Checkerboard Context Model for Efficient Learned Image Compressioncited by 202
  5. 2018Context-adaptive Entropy Model for End-to-end Optimized Image Compressioncited by 143
  6. 2020Nonlinear Transform Codingcited by 136
  7. 2019Variable Rate Deep Image Compression With a Conditional Autoencodercited by 114
  8. 2019Non-local Attention Optimized Deep Image Compressioncited by 92
  9. 2020Asymmetric Gained Deep Image Compression With Continuous Rate Adaptationcited by 85
  10. 2016End-to-end optimization of nonlinear transform codes for perceptual qualitycited by 73
  11. 2020Causal Contextual Prediction for Learned Image Compressioncited by 73
  12. 2021Variable-Rate Deep Image Compression through Spatially-Adaptive Feature Transformcited by 58
  13. 2021Soft then Hard: Rethinking the Quantization in Neural Image Compressioncited by 51
  14. 2020Learning Accurate Entropy Model with Global Reference for Image Compressioncited by 45
  15. 2019Efficient and Effective Context-Based Convolutional Entropy Modeling for Image Compressioncited by 33
  16. 2020Learned Block-based Hybrid Image Compressioncited by 30
  17. 2019Multi-scale and Context-adaptive Entropy Model for Image Compressioncited by 9
  18. 2020DeepQTMT: A Deep Learning Approach for Fast QTMT-based CU Partition of Intra-mode VVCcited by 6
  19. 2019Learned Fast HEVC Intra Codingcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Balanced Rate-Distortion Optimization in Learned Image Compressioncited by 5
  2. 2026Leveraging Second-Order Curvature for Efficient Learned Image Compression: Theory and Empirical Evidencecited by 2