# Efficient Learned Image Compression Through Knowledge Distillation

arXiv: https://arxiv.org/abs/2509.10366 · HTML: https://mlsanity.com/paper/2509.10366/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 13 of its 18 references that are in the map.
- First posted on arXiv 12 Sep 2025.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 13 of 13)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Channel-wise Autoregressive Entropy Models for Learned Image Compression](https://mlsanity.com/paper/2007.08739/) (2020)
- [End-to-end optimization of nonlinear transform codes for perceptual quality](https://mlsanity.com/paper/1607.05006/) (2016)
- [Cross-Architecture Knowledge Distillation](https://mlsanity.com/paper/2207.05273/) (2022)
- [Fast and High-Performance Learned Image Compression With Improved Checkerboard Context Model, Deformable Residual Module, and Knowledge Distillation](https://mlsanity.com/paper/2309.02529/) (2023)
- [Bjontegaard Delta (BD): A Tutorial Overview of the Metric, Evolution, Challenges, and Recommendations](https://mlsanity.com/paper/2401.04039/) (2024)
- [Microdosing: Knowledge Distillation for GAN based Compression](https://mlsanity.com/paper/2201.02624/) (2022)
- [Lightweight Embedded FPGA Deployment of Learned Image Compression with Knowledge Distillation and Hybrid Quantization](https://mlsanity.com/paper/2503.04832/) (2025)
- [Security and Real-time FPGA integration for Learned Image Compression](https://mlsanity.com/paper/2503.04867/) (2025)

## Built on by (most cited first, 1 of 1)

- [KD-NVC: A Search-and-Distill Framework to Accelerate Neural Video Coding](https://mlsanity.com/paper/2606.04595/) (2026)

## Abstract (arXiv)

Learned image compression sits at the intersection of machine learning and image processing. With advances in deep learning, neural network-based compression methods have emerged. In this process, an encoder maps the image to a low-dimensional latent space, which is then quantized, entropy-coded into a binary bitstream, and transmitted to the receiver. At the receiver end, the bitstream is entropy-decoded, and a decoder reconstructs an approximation of the original image. Recent research suggests that these models consistently outperform conventional codecs. However, they require significant processing power, making them unsuitable for real-time use on resource-constrained platforms, which hinders their deployment in mainstream applications. This study aims to reduce the resource requirements of neural networks used for image compression by leveraging knowledge distillation, a training paradigm where smaller neural networks, partially trained on the outputs of larger, more complex models, can achieve better performance than when trained independently. Our work demonstrates that knowledge distillation can be effectively applied to image compression tasks: i) across various architecture sizes, ii) to achieve different image quality/bit rate tradeoffs, and iii) to save processing and energy resources. This approach introduces new settings and hyperparameters, and future research could explore the impact of different teacher models, as well as alternative loss functions. Knowledge distillation could also be extended to transformer-based models. The code is publicly available at: https://github.com/FABallemand/PRIM .