# Image coding for machines: an end-to-end learned approach

arXiv: https://arxiv.org/abs/2108.09993 · HTML: https://mlsanity.com/paper/2108.09993/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 18 references that are in the map.
- First posted on arXiv 23 Aug 2021.

## Builds on (most cited first, 12 of 12)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Pixel Recurrent Neural Networks](https://mlsanity.com/paper/1601.06759/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Practical Full Resolution Learned Lossless Image Compression](https://mlsanity.com/paper/1811.12817/) (2018)
- [End-to-end optimization of nonlinear transform codes for perceptual quality](https://mlsanity.com/paper/1607.05006/) (2016)
- [Adapting JPEG XS gains and priorities to tasks and contents](https://mlsanity.com/paper/2005.08768/) (2020)

## Built on by (most cited first, 30 of 47)

- [Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities](https://mlsanity.com/paper/2210.06640/) (2022)
- [Image Coding for Machines with Omnipotent Feature Learning](https://mlsanity.com/paper/2207.01932/) (2022)
- [Towards End-to-End Image Compression and Analysis with Transformers](https://mlsanity.com/paper/2112.09300/) (2021)
- [A Coding Framework and Benchmark towards Low-Bitrate Video Understanding](https://mlsanity.com/paper/2202.02813/) (2022)
- [Image Coding for Machines with Edge Information Learning Using Segment Anything](https://mlsanity.com/paper/2403.04173/) (2024)
- [All-in-One Image Coding for Joint Human-Machine Vision with Multi-Path Aggregation](https://mlsanity.com/paper/2409.19660/) (2024)
- [Generative Visual Compression: A Review](https://mlsanity.com/paper/2402.02140/) (2024)
- [Semantically Structured Image Compression via Irregular Group-Based Decoupling](https://mlsanity.com/paper/2305.02586/) (2023)
- [Video Coding for Machine: Compact Visual Representation Compression for Intelligent Collaborative Analytics](https://mlsanity.com/paper/2110.09241/) (2021)
- [Rate-Distortion-Cognition Controllable Versatile Neural Image Compression](https://mlsanity.com/paper/2407.11700/) (2024)
- [Feature Compression for Rate Constrained Object Detection on the Edge](https://mlsanity.com/paper/2204.07314/) (2022)
- [Prompt-ICM: A Unified Framework towards Image Coding for Machines with Task-driven Prompts](https://mlsanity.com/paper/2305.02578/) (2023)
- [Image Coding for Machines with Object Region Learning](https://mlsanity.com/paper/2308.13984/) (2023)
- [NeuCo-Bench: A Novel Benchmark Framework for Neural Embeddings in Earth Observation](https://mlsanity.com/paper/2510.17914/) (2025)
- [Scalable Image Coding for Humans and Machines Using Feature Fusion Network](https://mlsanity.com/paper/2405.09152/) (2024)
- [High Efficiency Image Compression for Large Visual-Language Models](https://mlsanity.com/paper/2407.17060/) (2024)
- [Fostering the Robustness of White-Box Deep Neural Network Watermarks by Neuron Alignment](https://mlsanity.com/paper/2112.14108/) (2021)
- [An Efficient Adaptive Compression Method for Human Perception and Machine Vision Tasks](https://mlsanity.com/paper/2501.04329/) (2025)
- [Towards Point Cloud Compression for Machine Perception: A Simple and Strong Baseline by Learning the Octree Depth Level Predictor](https://mlsanity.com/paper/2406.00791/) (2024)
- [Unified Coding for Both Human Perception and Generalized Machine Analytics with CLIP Supervision](https://mlsanity.com/paper/2501.04579/) (2025)
- [Diff-ICMH: Harmonizing Machine and Human Vision in Image Compression with Generative Prior](https://mlsanity.com/paper/2511.22549/) (2025)
- [Bridging the gap between image coding for machines and humans](https://mlsanity.com/paper/2401.10732/) (2024)
- [Delta-ICM: Entropy Modeling with Delta Function for Learned Image Compression](https://mlsanity.com/paper/2410.07669/) (2024)
- [Explicit Residual-Based Scalable Image Coding for Humans and Machines](https://mlsanity.com/paper/2506.19297/) (2025)
- [Privacy-Preserving Feature Coding for Machines](https://mlsanity.com/paper/2210.00727/) (2022)
- [Channel-wise Feature Decorrelation for Enhanced Learned Image Compression](https://mlsanity.com/paper/2403.10936/) (2024)
- [Competitive Learning for Achieving Content-specific Filters in Video Coding for Machines](https://mlsanity.com/paper/2406.12367/) (2024)
- [DT-JRD: Deep Transformer based Just Recognizable Difference Prediction Model for Video Coding for Machines](https://mlsanity.com/paper/2411.09308/) (2024)
- [Guided Diffusion for the Extension of Machine Vision to Human Visual Perception](https://mlsanity.com/paper/2503.17907/) (2025)
- [Compression Tells Intelligence: Visual Coding, Visual Token Technology, and the Unification](https://mlsanity.com/paper/2601.20742/) (2026)

## Abstract (arXiv)

Over recent years, deep learning-based computer vision systems have been applied to images at an ever-increasing pace, oftentimes representing the only type of consumption for those images. Given the dramatic explosion in the number of images generated per day, a question arises: how much better would an image codec targeting machine-consumption perform against state-of-the-art codecs targeting human-consumption? In this paper, we propose an image codec for machines which is neural network (NN) based and end-to-end learned. In particular, we propose a set of training strategies that address the delicate problem of balancing competing loss functions, such as computer vision task losses, image distortion losses, and rate loss. Our experimental results show that our NN-based codec outperforms the state-of-the-art Versa-tile Video Coding (VVC) standard on the object detection and instance segmentation tasks, achieving -37.87% and -32.90% of BD-rate gain, respectively, while being fast thanks to its compact size. To the best of our knowledge, this is the first end-to-end learned machine-targeted image codec.