# Overfitting for Fun and Profit: Instance-Adaptive Data Compression

arXiv: https://arxiv.org/abs/2101.08687 · HTML: https://mlsanity.com/paper/2101.08687/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 32 references that are in the map.
- First posted on arXiv 21 Jan 2021.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [DVC: An End-to-end Deep Video Compression Framework](https://mlsanity.com/paper/1812.00101/) (2018)
- [Lossy Image Compression with Compressive Autoencoders](https://mlsanity.com/paper/1703.00395/) (2017)
- [Video Compression through Image Interpolation](https://mlsanity.com/paper/1804.06919/) (2018)
- [Inference Suboptimality in Variational Autoencoders](https://mlsanity.com/paper/1801.03558/) (2018)
- [Density Modeling of Images using a Generalized Normalization Transformation](https://mlsanity.com/paper/1511.06281/) (2015)
- [Bayesian Compression for Deep Learning](https://mlsanity.com/paper/1705.08665/) (2017)
- [Video Compression With Rate-Distortion Autoencoders](https://mlsanity.com/paper/1908.05717/) (2019)
- [Learned Video Compression](https://mlsanity.com/paper/1811.06981/) (2018)
- [Improving Inference for Neural Image Compression](https://mlsanity.com/paper/2006.04240/) (2020)
- [Content Adaptive and Error Propagation Aware Deep Video Compression](https://mlsanity.com/paper/2003.11282/) (2020)
- [Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameters](https://mlsanity.com/paper/1810.00440/) (2018)
- [Content Adaptive Optimization for Neural Image Compression](https://mlsanity.com/paper/1906.01223/) (2019)
- [Learned Video Compression via Joint Spatial-Temporal Correlation Exploration](https://mlsanity.com/paper/1912.06348/) (2019)
- [Efficient Adaptation of Neural Network Filter for Video Compression](https://mlsanity.com/paper/2007.14267/) (2020)
- [Taxonomy and Evaluation of Structured Compression of Convolutional Neural Networks](https://mlsanity.com/paper/1912.09802/) (2019)
- [Block-optimized Variable Bit Rate Neural Image Compression](https://mlsanity.com/paper/1805.10887/) (2018)
- [Compressing Weight-updates for Image Artifacts Removal Neural Networks](https://mlsanity.com/paper/1905.04079/) (2019)
- [Utilising Low Complexity CNNs to Lift Non-Local Redundancies in Video Coding](https://mlsanity.com/paper/1910.08737/) (2019)
- [Feedback Recurrent AutoEncoder](https://mlsanity.com/paper/1911.04018/) (2019)

## Built on by (most cited first, 30 of 38)

- [COIN: COmpression with Implicit Neural representations](https://mlsanity.com/paper/2103.03123/) (2021)
- [Implicit Neural Representations for Image Compression](https://mlsanity.com/paper/2112.04267/) (2021)
- [Towards Practical Real-Time Neural Video Compression](https://mlsanity.com/paper/2502.20762/) (2025)
- [C3: High-performance and low-complexity neural compression from a single image or video](https://mlsanity.com/paper/2312.02753/) (2023)
- [Implicit Neural Video Compression](https://mlsanity.com/paper/2112.11312/) (2021)
- [Neural Data-Dependent Transform for Learned Image Compression](https://mlsanity.com/paper/2203.04963/) (2022)
- [NVRC: Neural Video Representation Compression](https://mlsanity.com/paper/2409.07414/) (2024)
- [Computationally-Efficient Neural Image Compression with Shallow Decoders](https://mlsanity.com/paper/2304.06244/) (2023)
- [MobileCodec: Neural Inter-frame Video Compression on Mobile Devices](https://mlsanity.com/paper/2207.08338/) (2022)
- [PNVC: Towards Practical INR-based Video Compression](https://mlsanity.com/paper/2409.00953/) (2024)
- [Instance-Adaptive Video Compression: Improving Neural Codecs by Training on the Test Set](https://mlsanity.com/paper/2111.10302/) (2021)
- [Content Adaptive Latents and Decoder for Neural Image Compression](https://mlsanity.com/paper/2212.10132/) (2022)
- [MobileNVC: Real-time 1080p Neural Video Compression on a Mobile Device](https://mlsanity.com/paper/2310.01258/) (2023)
- [Universal Deep Image Compression via Content-Adaptive Optimization with Adapters](https://mlsanity.com/paper/2211.00918/) (2022)
- [Dynamic Low-Rank Instance Adaptation for Universal Neural Image Compression](https://mlsanity.com/paper/2308.07733/) (2023)
- [GIViC: Generative Implicit Video Compression](https://mlsanity.com/paper/2503.19604/) (2025)
- [Bit Allocation using Optimization](https://mlsanity.com/paper/2209.09422/) (2022)
- [Toward Adaptive Semantic Communications: Efficient Data Transmission via Online Learned Nonlinear Transform Source-Channel Coding](https://mlsanity.com/paper/2211.04339/) (2022)
- [Reducing The Amortization Gap of Entropy Bottleneck In End-to-End Image Compression](https://mlsanity.com/paper/2209.00964/) (2022)
- [Adaptation and Attention for Neural Video Coding](https://mlsanity.com/paper/2112.08767/) (2021)
- [Boosting neural video codecs by exploiting hierarchical redundancy](https://mlsanity.com/paper/2208.04303/) (2022)
- [Microdosing: Knowledge Distillation for GAN based Compression](https://mlsanity.com/paper/2201.02624/) (2022)
- [Low-Latency Neural Stereo Streaming](https://mlsanity.com/paper/2403.17879/) (2024)
- [Test-time adaptation for image compression with distribution regularization](https://mlsanity.com/paper/2410.12191/) (2024)
- [Improving The Reconstruction Quality by Overfitted Decoder Bias in Neural Image Compression](https://mlsanity.com/paper/2210.04898/) (2022)
- [Reducing The Mismatch Between Marginal and Learned Distributions in Neural Video Compression](https://mlsanity.com/paper/2210.06596/) (2022)
- [Robustly overfitting latents for flexible neural image compression](https://mlsanity.com/paper/2401.17789/) (2024)
- [Survey on Visual Signal Coding and Processing with Generative Models: Technologies, Standards and Optimization](https://mlsanity.com/paper/2405.14221/) (2024)
- [Hyperspectral Image Compression Using Implicit Neural Representation](https://mlsanity.com/paper/2302.04129/) (2023)
- [Learned Compression of Encoding Distributions](https://mlsanity.com/paper/2406.13059/) (2024)

## Abstract (arXiv)

Neural data compression has been shown to outperform classical methods in terms of RD performance, with results still improving rapidly. At a high level, neural compression is based on an autoencoder that tries to reconstruct the input instance from a (quantized) latent representation, coupled with a prior that is used to losslessly compress these latents. Due to limitations on model capacity and imperfect optimization and generalization, such models will suboptimally compress test data in general. However, one of the great strengths of learned compression is that if the test-time data distribution is known and relatively low-entropy (e.g. a camera watching a static scene, a dash cam in an autonomous car, etc.), the model can easily be finetuned or adapted to this distribution, leading to improved RD performance. In this paper we take this concept to the extreme, adapting the full model to a single video, and sending model updates (quantized and compressed using a parameter-space prior) along with the latent representation. Unlike previous work, we finetune not only the encoder/latents but the entire model, and - during finetuning - take into account both the effect of model quantization and the additional costs incurred by sending the model updates. We evaluate an image compression model on I-frames (sampled at 2 fps) from videos of the Xiph dataset, and demonstrate that full-model adaptation improves RD performance by ~1 dB, with respect to encoder-only finetuning.