End-to-end optimization of nonlinear transform codes for perceptual quality
Not on the map. Why?
Abstract (arXiv)
We introduce a general framework for end-to-end optimization of the rate--distortion performance of nonlinear transform codes assuming scalar quantization. The framework can be used to optimize any differentiable pair of analysis and synthesis transforms in combination with any differentiable perceptual metric. As an example, we consider a code built from a linear transform followed by a form of multi-dimensional local gain control. Distortion is measured with a state-of-the-art perceptual metric. When optimized over a large database of images, this representation offers substantial improvements in bitrate and perceptual appearance over fixed (DCT) codes, and over linear transform codes optimized for mean squared error.
Key facts. Cited by 72 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Jul 2016.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 15; 2022: 17; 2023: 13; 2024: 13; 2025: 10; 2026: 4.
72 papers.
Sort:
- 2021Enhanced Invertible Encoding for Learned Image Compressioncited by 102
- 2022Entroformer: A Transformer-based Entropy Model for Learned Image Compressioncited by 102
- 2022An Introduction to Neural Data Compressioncited by 78
- 2024End-to-End Rate-Distortion Optimized 3D Gaussian Representationcited by 71
- 2021ELF-VC: Efficient Learned Flexible-Rate Video Codingcited by 68
- 2024Learning by Reconstruction Produces Uninformative Features For Perceptioncited by 54
- 2021Image coding for machines: an end-to-end learned approachcited by 47
- 2021Learned Image Compression with Gaussian-Laplacian-Logistic Mixture Model and Concatenated Residual Modulescited by 33
- 2025Joint Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self Supervised Learningcited by 33
- 2022Lossy Image Compression with Quantized Hierarchical VAEscited by 32
- 2021Extending Neural P-frame Codecs for B-frame Codingcited by 30
- 2022AlphaVC: High-Performance and Efficient Learned Video Compressioncited by 28
38 more
- 2021On Perceptual Lossy Compression: The Cost of Perceptual Reconstruction and An Optimal Training Frameworkcited by 27
- 20254DGC: Rate-Aware 4D Gaussian Compression for Efficient Streamable Free-Viewpoint Videocited by 26
- 2023Computationally-Efficient Neural Image Compression with Shallow Decoderscited by 22
- 2021LVAC: Learned Volumetric Attribute Compression for Point Clouds using Coordinate Based Networkscited by 19
- 2022MobileCodec: Neural Inter-frame Video Compression on Mobile Devicescited by 15
- 2023M2T: Masking Transformers Twice for Faster Decodingcited by 14
- 2023Frequency Disentangled Features in Neural Image Compressioncited by 14
- 2021IICNet: A Generic Framework for Reversible Image Conversioncited by 13
- 2024HPC: Hierarchical Progressive Coding Framework for Volumetric Videocited by 13
- 2022Scale-arbitrary Invertible Image Downscalingcited by 11
- 2025Lossy Neural Compression for Geospatial Analytics: A Reviewcited by 10
- 2021Insights from Generative Modeling for Neural Video Compressioncited by 9
- 2021Image Compression with Recurrent Neural Network and Generalized Divisive Normalizationcited by 9
- 2021Self-Organized Variational Autoencoders (Self-VAE) for Learned Image Compressioncited by 8
- 2022Attention-Based Generative Neural Image Compression on Solar Dynamics Observatorycited by 8
- 2024Generalized Gaussian Model for Learned Image Compressioncited by 8
- 2021Rate Distortion Characteristic Modeling for Neural Image Compressioncited by 6
- 2021OSLO: On-the-Sphere Learning for Omnidirectional images and its application to 360-degree image compressioncited by 6
- 2022Optimizing Image Compression via Joint Learning with Denoisingcited by 6
- 2022IDLat: An Importance-Driven Latent Generation Method for Scientific Datacited by 6
- 2022Learned Image Compression with Generalized Octave Convolution and Cross-Resolution Parameter Estimationcited by 6
- 2023Sandwiched Video Compression: Efficiently Extending the Reach of Standard Codecs with Neural Wrapperscited by 6
- 2023Exploring Resolution Fields for Scalable Image Compression with Uncertainty Guidancecited by 6
- 2022Reducing Redundancy in the Bottleneck Representation of the Autoencoderscited by 5
- 2024Uncertainty-Aware Deep Video Compression with Ensemblescited by 5
- 2022Variable Rate Compression for Raw 3D Point Cloudscited by 4
- 2022Video Coding Using Learned Latent GAN Compressioncited by 4
- 2024Neural NeRF Compressioncited by 4
- 2023Volumetric Attribute Compression for 3D Point Clouds using Feedforward Network with Geometric Attentioncited by 3
- 2023Make Lossy Compression Meaningful for Low-Light Imagescited by 3
- 2022Recognition-Aware Learned Image Compressioncited by 2
- 2023Exploring the Rate-Distortion-Complexity Optimization in Neural Image Compressioncited by 2
- 2023GAN-based Image Compression with Improved RDO Processcited by 2
- 20253DM-WeConvene: Learned Image Compression with 3D Multi-Level Wavelet-Domain Convolution and Entropy Modelcited by 2
- 2021Principal Bit Analysis: Autoencoding with Schur-Concave Losscited by 1
- 2023Multi-spectral Entropy Constrained Neural Compression of Solar Imagerycited by 1
- 2023CPIPS: Learning to Preserve Perceptual Distances in End-to-End Image Compressioncited by 1
- 2023Neural-based Compression Scheme for Solar Image Datacited by 1
50 most cited of 72 All 72 (JSON)
- 2026Lossy Compressive Text Autoencoders7 Oct 2026
- 2026ARCHE: Autoregressive Residual Compression with Hyperprior and Excitation10 Mar 2026
- 2026Understanding Rate-Distortion Performance in Distributed Transformer Inference29 Jan 2026
- 2026Efficient training for compact compression models via sequential distillation9 Jan 2026
- 2025Spectral and Spatial Graph Learning for Multispectral Solar Image Compression30 Dec 2025
- 2025ProDAT: Progressive Density-Aware Tail-Drop for Point Cloud Coding20 Oct 2025
- 2025Efficient Learned Image Compression Through Knowledge Distillation12 Sep 2025
- 2025Optimizing Learned Image Compression on Scalar and Entropy-Constraint Quantization10 Jun 2025
- 2025Joint Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self Supervised Learning18 May 2025
- 20253DM-WeConvene: Learned Image Compression with 3D Multi-Level Wavelet-Domain Convolution and Entropy Model7 Apr 2025
- 2025Bitstream Collisions in Neural Image Compression via Adversarial Perturbations25 Mar 2025
- 20254DGC: Rate-Aware 4D Gaussian Compression for Efficient Streamable Free-Viewpoint Video24 Mar 2025
38 more
- 2025Lossy Neural Compression for Geospatial Analytics: A Review3 Mar 2025
- 2025Interleaved Block-based Learned Image Compression with Feature Enhancement and Quantization Error Compensation21 Feb 2025
- 2024Generalized Gaussian Model for Learned Image Compression28 Nov 2024
- 2024Enhancing Learned Image Compression via Cross Window-based Attention28 Oct 2024
- 2024Learned Compression of Point Cloud Geometry and Attributes in a Single Model through Multimodal Rate-Control1 Aug 2024
- 2024Neural-based Video Compression on Solar Dynamics Observatory Images12 Jul 2024
- 2024HPC: Hierarchical Progressive Coding Framework for Volumetric Video12 Jul 2024
- 2024MNeRV: A Multilayer Neural Representation for Videos10 Jul 2024
- 2024Neural NeRF Compression13 Jun 2024
- 2024End-to-End Rate-Distortion Optimized 3D Gaussian Representation9 Apr 2024
- 2024Uncertainty-Aware Deep Video Compression with Ensembles28 Mar 2024
- 2024Neural Network Assisted Lifting Steps For Improved Fully Scalable Lossy Image Compression in JPEG 20004 Mar 2024
- 2024Traditional Transformation Theory Guided Model for Learned Image Compression24 Feb 2024
- 2024Learning by Reconstruction Produces Uninformative Features For Perception17 Feb 2024
- 2024Perceptual Learned Image Compression via End-to-End JND-Based Optimization5 Feb 2024
- 2023Neural-based Compression Scheme for Solar Image Data6 Nov 2023
- 2023CPIPS: Learning to Preserve Perceptual Distances in End-to-End Image Compression1 Oct 2023
- 2023Multi-spectral Entropy Constrained Neural Compression of Solar Imagery19 Sep 2023
- 2023Implicit Neural Multiple Description for DNA-based data storage13 Sep 2023
- 2023Frequency Disentangled Features in Neural Image Compression4 Aug 2023
- 2023GAN-based Image Compression with Improved RDO Process18 Jun 2023
- 2023Exploring Resolution Fields for Scalable Image Compression with Uncertainty Guidance15 Jun 2023
- 2023Make Lossy Compression Meaningful for Low-Light Images24 May 2023
- 2023Exploring the Rate-Distortion-Complexity Optimization in Neural Image Compression12 May 2023
- 2023M2T: Masking Transformers Twice for Faster Decoding14 Apr 2023
- 2023Computationally-Efficient Neural Image Compression with Shallow Decoders13 Apr 2023
- 2023Volumetric Attribute Compression for 3D Point Clouds using Feedforward Network with Geometric Attention1 Apr 2023
- 2023Sandwiched Video Compression: Efficiently Extending the Reach of Standard Codecs with Neural Wrappers20 Mar 2023
- 2022Attention-Based Generative Neural Image Compression on Solar Dynamics Observatory12 Oct 2022
- 2022Learned Image Compression with Generalized Octave Convolution and Cross-Resolution Parameter Estimation7 Sep 2022
- 2022Learned Lossless Image Compression With Combined Autoregressive Models And Attention Modules30 Aug 2022
- 2022Lossy Image Compression with Quantized Hierarchical VAEs27 Aug 2022
- 2022IDLat: An Importance-Driven Latent Generation Method for Scientific Data5 Aug 2022
- 2022AlphaVC: High-Performance and Efficient Learned Video Compression29 Jul 2022
- 2022Optimizing Image Compression via Joint Learning with Denoising22 Jul 2022
- 2022MobileCodec: Neural Inter-frame Video Compression on Mobile Devices18 Jul 2022
- 2022Video Coding Using Learned Latent GAN Compression9 Jul 2022
- 2022Asymmetric Learned Image Compression with Multi-Scale Residual Block, Importance Map, and Post-Quantization Filtering21 Jun 2022
50 newest of 72 All 72 (JSON)
- 2021Enhanced Invertible Encoding for Learned Image Compressioncited by 102
- 2022Entroformer: A Transformer-based Entropy Model for Learned Image Compressioncited by 102
- 2022An Introduction to Neural Data Compressioncited by 78
- 2024End-to-End Rate-Distortion Optimized 3D Gaussian Representationcited by 71
- 2021ELF-VC: Efficient Learned Flexible-Rate Video Codingcited by 68
- 2024Learning by Reconstruction Produces Uninformative Features For Perceptioncited by 54
- 2021Image coding for machines: an end-to-end learned approachcited by 47
- 2021Learned Image Compression with Gaussian-Laplacian-Logistic Mixture Model and Concatenated Residual Modulescited by 33
- 2025Joint Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self Supervised Learningcited by 33
- 2022Lossy Image Compression with Quantized Hierarchical VAEscited by 32
- 2021Extending Neural P-frame Codecs for B-frame Codingcited by 30
- 2022AlphaVC: High-Performance and Efficient Learned Video Compressioncited by 28
12 most cited of 72 All 72 (JSON)
- 2026Lossy Compressive Text Autoencoders7 Oct 2026
- 2026ARCHE: Autoregressive Residual Compression with Hyperprior and Excitation10 Mar 2026
- 2026Understanding Rate-Distortion Performance in Distributed Transformer Inference29 Jan 2026
- 2026Efficient training for compact compression models via sequential distillation9 Jan 2026
- 2025Spectral and Spatial Graph Learning for Multispectral Solar Image Compression30 Dec 2025
- 2025ProDAT: Progressive Density-Aware Tail-Drop for Point Cloud Coding20 Oct 2025
- 2025Efficient Learned Image Compression Through Knowledge Distillation12 Sep 2025
- 2025Optimizing Learned Image Compression on Scalar and Entropy-Constraint Quantization10 Jun 2025
- 2025Joint Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self Supervised Learning18 May 2025
- 20253DM-WeConvene: Learned Image Compression with 3D Multi-Level Wavelet-Domain Convolution and Entropy Model7 Apr 2025
- 2025Bitstream Collisions in Neural Image Compression via Adversarial Perturbations25 Mar 2025
- 20254DGC: Rate-Aware 4D Gaussian Compression for Efficient Streamable Free-Viewpoint Video24 Mar 2025
12 newest of 72 All 72 (JSON)