# Neural Data-Dependent Transform for Learned Image Compression

arXiv: https://arxiv.org/abs/2203.04963 · HTML: https://mlsanity.com/paper/2203.04963/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 35 references that are in the map.
- First posted on arXiv 9 Mar 2022.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [Single Image Super-Resolution via a Holistic Attention Network](https://mlsanity.com/paper/2008.08767/) (2020)
- [Context-adaptive Entropy Model for End-to-end Optimized Image Compression](https://mlsanity.com/paper/1809.10452/) (2018)
- [Nonlinear Transform Coding](https://mlsanity.com/paper/2007.03034/) (2020)
- [Density Modeling of Images using a Generalized Normalization Transformation](https://mlsanity.com/paper/1511.06281/) (2015)
- [Non-local Attention Optimized Deep Image Compression](https://mlsanity.com/paper/1904.09757/) (2019)
- [Content Adaptive and Error Propagation Aware Deep Video Compression](https://mlsanity.com/paper/2003.11282/) (2020)
- [Learning End-to-End Lossy Image Compression: A Benchmark](https://mlsanity.com/paper/2002.03711/) (2020)
- [Overfitting for Fun and Profit: Instance-Adaptive Data Compression](https://mlsanity.com/paper/2101.08687/) (2021)
- [Instance-Adaptive Video Compression: Improving Neural Codecs by Training on the Test Set](https://mlsanity.com/paper/2111.10302/) (2021)
- [Image-Dependent Local Entropy Models for Learned Image Compression](https://mlsanity.com/paper/1805.12295/) (2018)
- [Spatially adaptive image compression using a tiled deep network](https://mlsanity.com/paper/1802.02629/) (2018)

## Built on by (most cited first, 30 of 30)

- [Learned Image Compression with Mixed Transformer-CNN Architectures](https://mlsanity.com/paper/2303.14978/) (2023)
- [Lossy Image Compression with Conditional Diffusion Models](https://mlsanity.com/paper/2209.06950/) (2022)
- [MLIC: Multi-Reference Entropy Model for Learned Image Compression](https://mlsanity.com/paper/2211.07273/) (2022)
- [MLIC++: Linear Complexity Multi-Reference Entropy Modeling for Learned Image Compression](https://mlsanity.com/paper/2307.15421/) (2023)
- [Lossy Image Compression with Quantized Hierarchical VAEs](https://mlsanity.com/paper/2208.13056/) (2022)
- [Learned Image Compression with Dictionary-based Entropy Model](https://mlsanity.com/paper/2504.00496/) (2025)
- [MambaVC: Learned Visual Compression with Selective State Spaces](https://mlsanity.com/paper/2405.15413/) (2024)
- [HQ-VAE: Hierarchical Discrete Representation Learning with Variational Bayes](https://mlsanity.com/paper/2401.00365/) (2023)
- [Causal Context Adjustment Loss for Learned Image Compression](https://mlsanity.com/paper/2410.04847/) (2024)
- [SC2 Benchmark: Supervised Compression for Split Computing](https://mlsanity.com/paper/2203.08875/) (2022)
- [Group-aware Parameter-efficient Updating for Content-Adaptive Neural Video Compression](https://mlsanity.com/paper/2405.04274/) (2024)
- [High-Fidelity Variable-Rate Image Compression via Invertible Activation Transformation](https://mlsanity.com/paper/2209.05054/) (2022)
- [Toward Adaptive Semantic Communications: Efficient Data Transmission via Online Learned Nonlinear Transform Source-Channel Coding](https://mlsanity.com/paper/2211.04339/) (2022)
- [Region-Adaptive Transform with Segmentation Prior for Image Compression](https://mlsanity.com/paper/2403.00628/) (2024)
- [LLIC: Large Receptive Field Transform Coding with Adaptive Weights for Learned Image Compression](https://mlsanity.com/paper/2304.09571/) (2023)
- [Another Way to the Top: Exploit Contextual Clustering in Learned Image Coding](https://mlsanity.com/paper/2401.11615/) (2024)
- [3D Gaussian Splatting Data Compression with Mixture of Priors](https://mlsanity.com/paper/2505.03310/) (2025)
- [An Efficient Adaptive Compression Method for Human Perception and Machine Vision Tasks](https://mlsanity.com/paper/2501.04329/) (2025)
- [STanH : Parametric Quantization for Variable Rate Learned Image Compression](https://mlsanity.com/paper/2410.00557/) (2024)
- [Make Lossy Compression Meaningful for Low-Light Images](https://mlsanity.com/paper/2305.15030/) (2023)
- [Laplacian-guided Entropy Model in Neural Codec with Blur-dissipated Synthesis](https://mlsanity.com/paper/2403.16258/) (2024)
- [Exploring Effective Mask Sampling Modeling for Neural Image Compression](https://mlsanity.com/paper/2306.05704/) (2023)
- [Coding for Intelligence from the Perspective of Category](https://mlsanity.com/paper/2407.01017/) (2024)
- [Bi-Level Spatial and Channel-aware Transformer for Learned Image Compression](https://mlsanity.com/paper/2408.03842/) (2024)
- [MatSpectNet: Material Segmentation Network with Domain-Aware and Physically-Constrained Hyperspectral Reconstruction](https://mlsanity.com/paper/2307.11466/) (2023)
- [Dynamic Kernel-Based Adaptive Spatial Aggregation for Learned Image Compression](https://mlsanity.com/paper/2308.08723/) (2023)
- [LSCD: A Large-Scale Screen Content Dataset for Video Compression](https://mlsanity.com/paper/2308.09332/) (2023)
- [Accelerating Learned Video Compression via Low-Resolution Representation Learning](https://mlsanity.com/paper/2407.16418/) (2024)
- [Edge-based Denoising Image Compression](https://mlsanity.com/paper/2409.10978/) (2024)
- [End-to-End Semantic Preservation in Text-Aware Image Compression Systems](https://mlsanity.com/paper/2503.19495/) (2025)

## Abstract (arXiv)

Learned image compression has achieved great success due to its excellent modeling capacity, but seldom further considers the Rate-Distortion Optimization (RDO) of each input image. To explore this potential in the learned codec, we make the first attempt to build a neural data-dependent transform and introduce a continuous online mode decision mechanism to jointly optimize the coding efficiency for each individual image. Specifically, apart from the image content stream, we employ an additional model stream to generate the transform parameters at the decoder side. The presence of a model stream enables our model to learn more abstract neural-syntax, which helps cluster the latent representations of images more compactly. Beyond the transform stage, we also adopt neural-syntax based post-processing for the scenarios that require higher quality reconstructions regardless of extra decoding overhead. Moreover, the involvement of the model stream further makes it possible to optimize both the representation and the decoder in an online way, i.e. RDO at the testing time. It is equivalent to a continuous online mode decision, like coding modes in the traditional codecs, to improve the coding efficiency based on the individual input image. The experimental results show the effectiveness of the proposed neural-syntax design and the continuous online mode decision mechanism, demonstrating the superiority of our method in coding efficiency compared to the latest conventional standard Versatile Video Coding (VVC) and other state-of-the-art learning-based methods.