# End-to-end optimized image compression for multiple machine tasks

arXiv: https://arxiv.org/abs/2103.04178 · HTML: https://mlsanity.com/paper/2103.04178/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 24 references that are in the map.
- First posted on arXiv 6 Mar 2021.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Taskonomy: Disentangling Task Transfer Learning](https://mlsanity.com/paper/1804.08328/) (2018)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [CNN Features off-the-shelf: an Astounding Baseline for Recognition](https://mlsanity.com/paper/1403.6382/) (2014)
- [Video Coding for Machines: A Paradigm of Collaborative Compression and Intelligent Analytics](https://mlsanity.com/paper/2001.03569/) (2020)
- [End-to-end Learning of Compressible Features](https://mlsanity.com/paper/2007.11797/) (2020)
- [End-to-end optimized image compression for machines, a study](https://mlsanity.com/paper/2011.06409/) (2020)
- [The Rate-Distortion-Accuracy Tradeoff: JPEG Case Study](https://mlsanity.com/paper/2008.00605/) (2020)

## Built on by (most cited first, 9 of 9)

- [All-in-One Image Coding for Joint Human-Machine Vision with Multi-Path Aggregation](https://mlsanity.com/paper/2409.19660/) (2024)
- [Generative Visual Compression: A Review](https://mlsanity.com/paper/2402.02140/) (2024)
- [Perceptual Video Coding for Machines via Satisfied Machine Ratio Modeling](https://mlsanity.com/paper/2211.06797/) (2022)
- [NeuCo-Bench: A Novel Benchmark Framework for Neural Embeddings in Earth Observation](https://mlsanity.com/paper/2510.17914/) (2025)
- [Identity Preserving Loss for Learned Image Compression](https://mlsanity.com/paper/2204.10869/) (2022)
- [Learning on JPEG-LDPC Compressed Images: Classifying with Syndromes](https://mlsanity.com/paper/2403.10202/) (2024)
- [All-in-One Transferring Image Compression from Human Perception to Multi-Machine Perception](https://mlsanity.com/paper/2504.12997/) (2025)
- [Lossy Common Information in a Learnable Gray-Wyner Network](https://mlsanity.com/paper/2601.21424/) (2026)
- [Goal-Oriented Source Coding using LDPC Codes for Compressed-Domain Image Classification](https://mlsanity.com/paper/2503.11954/) (2025)

## Abstract (arXiv)

An increasing share of captured images and videos are transmitted for storage and remote analysis by computer vision algorithms, rather than to be viewed by humans. Contrary to traditional standard codecs with engineered tools, neural network based codecs can be trained end-to-end to optimally compress images with respect to a target rate and any given differentiable performance metric. Although it is possible to train such compression tools to achieve better rate-accuracy performance for a particular computer vision task, it could be practical and relevant to re-use the compressed bit-stream for multiple machine tasks. For this purpose, we introduce 'Connectors' that are inserted between the decoder and the task algorithms to enable a direct transformation of the compressed content, which was previously optimized for a specific task, to multiple other machine tasks. We demonstrate the effectiveness of the proposed method by achieving significant rate-accuracy performance improvement for both image classification and object segmentation, using the same bit-stream, originally optimized for object detection.