# End-to-end Optimized Image Compression

arXiv: https://arxiv.org/abs/1611.01704 · HTML: https://mlsanity.com/paper/1611.01704/

## Key facts

- Cited by 696 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Nov 2016.

## Built on by (most cited first, 30 of 696)

- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [Finite Scalar Quantization: VQ-VAE Made Simple](https://mlsanity.com/paper/2309.15505/) (2023)
- [ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Coding](https://mlsanity.com/paper/2203.10886/) (2022)
- [Deep Contextual Video Compression](https://mlsanity.com/paper/2109.15047/) (2021)
- [Learned Image Compression with Mixed Transformer-CNN Architectures](https://mlsanity.com/paper/2303.14978/) (2023)
- [Checkerboard Context Model for Efficient Learned Image Compression](https://mlsanity.com/paper/2103.15306/) (2021)
- [From data to functa: Your data point is a function and you can treat it like one](https://mlsanity.com/paper/2201.12204/) (2022)
- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [Implicit Neural Representations for Image Compression](https://mlsanity.com/paper/2112.04267/) (2021)
- [FVC: A New Framework towards Deep Video Compression in Feature Space](https://mlsanity.com/paper/2105.09600/) (2021)
- [The Devil Is in the Details: Window-based Attention for Image Compression](https://mlsanity.com/paper/2203.08450/) (2022)
- [Hybrid Spatial-Temporal Entropy Modelling for Neural Video Compression](https://mlsanity.com/paper/2207.05894/) (2022)
- [Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges](https://mlsanity.com/paper/2103.04505/) (2021)
- [Entroformer: A Transformer-based Entropy Model for Learned Image Compression](https://mlsanity.com/paper/2202.05492/) (2022)
- [Enhanced Invertible Encoding for Learned Image Compression](https://mlsanity.com/paper/2108.03690/) (2021)
- [Towards image compression with perfect realism at ultra-low bitrates](https://mlsanity.com/paper/2310.10325/) (2023)
- [GaussianImage: 1000 FPS Image Representation and Compression by 2D Gaussian Splatting](https://mlsanity.com/paper/2403.08551/) (2024)
- [Improving Statistical Fidelity for Neural Image Compression with Implicit Local Likelihood Models](https://mlsanity.com/paper/2301.11189/) (2023)
- [Semantic Communications: Principles and Challenges](https://mlsanity.com/paper/2201.01389/) (2021)
- [Lossy Compression with Gaussian Diffusion](https://mlsanity.com/paper/2206.08889/) (2022)
- [Graph Barlow Twins: A self-supervised representation learning framework for graphs](https://mlsanity.com/paper/2106.02466/) (2021)
- [MLIC: Multi-Reference Entropy Model for Learned Image Compression](https://mlsanity.com/paper/2211.07273/) (2022)
- [COIN++: Neural Compression Across Modalities](https://mlsanity.com/paper/2201.12904/) (2022)
- [CompGS: Efficient 3D Scene Representation via Compressed Gaussian Splatting](https://mlsanity.com/paper/2404.09458/) (2024)
- [Improving the Diffusability of Autoencoders](https://mlsanity.com/paper/2502.14831/) (2025)
- [An Introduction to Neural Data Compression](https://mlsanity.com/paper/2202.06533/) (2022)
- [Nonlinear Transform Source-Channel Coding for Semantic Communications](https://mlsanity.com/paper/2112.10961/) (2021)
- [End-to-End Rate-Distortion Optimized 3D Gaussian Representation](https://mlsanity.com/paper/2406.01597/) (2024)
- [Text + Sketch: Image Compression at Ultra Low Rates](https://mlsanity.com/paper/2307.01944/) (2023)

## Abstract (arXiv)

We describe an image compression method, consisting of a nonlinear analysis transformation, a uniform quantizer, and a nonlinear synthesis transformation. The transforms are constructed in three successive stages of convolutional linear filters and nonlinear activation functions. Unlike most convolutional neural networks, the joint nonlinearity is chosen to implement a form of local gain control, inspired by those used to model biological neurons. Using a variant of stochastic gradient descent, we jointly optimize the entire model for rate-distortion performance over a database of training images, introducing a continuous proxy for the discontinuous loss function arising from the quantizer. Under certain conditions, the relaxed loss function may be interpreted as the log likelihood of a generative model, as implemented by a variational autoencoder. Unlike these models, however, the compression model must operate at any given point along the rate-distortion curve, as specified by a trade-off parameter. Across an independent set of test images, we find that the optimized method generally exhibits better rate-distortion performance than the standard JPEG and JPEG 2000 compression methods. More importantly, we observe a dramatic improvement in visual quality for all images at all bit rates, which is supported by objective quality estimates using MS-SSIM.