# Rate-Distortion in Image Coding for Machines

arXiv: https://arxiv.org/abs/2209.11694 · HTML: https://mlsanity.com/paper/2209.11694/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 21 references that are in the map.
- First posted on arXiv 21 Sep 2022.

## Builds on (most cited first, 10 of 10)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [Scalable Image Coding for Humans and Machines](https://mlsanity.com/paper/2107.08373/) (2021)
- [Deep feature compression for collaborative object detection](https://mlsanity.com/paper/1802.03931/) (2018)
- [Multi-task learning with compressible features for Collaborative Intelligence](https://mlsanity.com/paper/1902.05179/) (2019)
- [Near-Lossless Deep Feature Compression for Collaborative Intelligence](https://mlsanity.com/paper/1804.09963/) (2018)

## Built on by (most cited first, 6 of 6)

- [All-in-One Image Coding for Joint Human-Machine Vision with Multi-Path Aggregation](https://mlsanity.com/paper/2409.19660/) (2024)
- [Learned Scalable Video Coding For Humans and Machines](https://mlsanity.com/paper/2307.08978/) (2023)
- [VVC+M: Plug and Play Scalable Image Coding for Humans and Machines](https://mlsanity.com/paper/2305.10453/) (2023)
- [Feature Coding for Scalable Machine Vision](https://mlsanity.com/paper/2512.10209/) (2025)
- [Improving Image Coding for Machines through Optimizing Encoder via Auxiliary Loss](https://mlsanity.com/paper/2402.08267/) (2024)
- [MoECodec: Image Compression for joint human and machine perception via Mixture-of-Experts](https://mlsanity.com/paper/2606.21033/) (2026)

## Abstract (arXiv)

In recent years, there has been a sharp increase in transmission of images to remote servers specifically for the purpose of computer vision. In many applications, such as surveillance, images are mostly transmitted for automated analysis, and rarely seen by humans. Using traditional compression for this scenario has been shown to be inefficient in terms of bit-rate, likely due to the focus on human based distortion metrics. Thus, it is important to create specific image coding methods for joint use by humans and machines. One way to create the machine side of such a codec is to perform feature matching of some intermediate layer in a Deep Neural Network performing the machine task. In this work, we explore the effects of the layer choice used in training a learnable codec for humans and machines. We prove, using the data processing inequality, that matching features from deeper layers is preferable in the sense of rate-distortion. Next, we confirm our findings empirically by re-training an existing model for scalable human-machine coding. In our experiments we show the trade-off between the human and machine sides of such a scalable model, and discuss the benefit of using deeper layers for training in that regard.