# Scalable Image Coding for Humans and Machines

arXiv: https://arxiv.org/abs/2107.08373 · HTML: https://mlsanity.com/paper/2107.08373/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jul 2021.

## Built on by (most cited first, 30 of 66)

- [Image Compression for Machine and Human Vision with Spatial-Frequency Adaptation](https://mlsanity.com/paper/2407.09853/) (2024)
- [A Coding Framework and Benchmark towards Low-Bitrate Video Understanding](https://mlsanity.com/paper/2202.02813/) (2022)
- [Image Coding for Machines with Edge Information Learning Using Segment Anything](https://mlsanity.com/paper/2403.04173/) (2024)
- [All-in-One Image Coding for Joint Human-Machine Vision with Multi-Path Aggregation](https://mlsanity.com/paper/2409.19660/) (2024)
- [Generative Visual Compression: A Review](https://mlsanity.com/paper/2402.02140/) (2024)
- [SMC++: Masked Learning of Unsupervised Video Semantic Compression](https://mlsanity.com/paper/2406.04765/) (2024)
- [Video Coding for Machine: Compact Visual Representation Compression for Intelligent Collaborative Analytics](https://mlsanity.com/paper/2110.09241/) (2021)
- [Scalable Face Image Coding via StyleGAN Prior: Towards Compression for Human-Machine Collaborative Vision](https://mlsanity.com/paper/2312.15622/) (2023)
- [Rate-Distortion-Cognition Controllable Versatile Neural Image Compression](https://mlsanity.com/paper/2407.11700/) (2024)
- [Semantic Segmentation in Learned Compressed Domain](https://mlsanity.com/paper/2209.01355/) (2022)
- [Scalable Video Coding for Humans and Machines](https://mlsanity.com/paper/2208.02512/) (2022)
- [Joint Image Compression and Denoising via Latent-Space Scalability](https://mlsanity.com/paper/2205.01874/) (2022)
- [Perceptual Video Coding for Machines via Satisfied Machine Ratio Modeling](https://mlsanity.com/paper/2211.06797/) (2022)
- [Rate-Distortion in Image Coding for Machines](https://mlsanity.com/paper/2209.11694/) (2022)
- [Exploring Resolution Fields for Scalable Image Compression with Uncertainty Guidance](https://mlsanity.com/paper/2306.08941/) (2023)
- [Task-Aware Encoder Control for Deep Video Compression](https://mlsanity.com/paper/2404.04848/) (2024)
- [Scalable Image Coding for Humans and Machines Using Feature Fusion Network](https://mlsanity.com/paper/2405.09152/) (2024)
- [Learned Scalable Video Coding For Humans and Machines](https://mlsanity.com/paper/2307.08978/) (2023)
- [An Efficient Adaptive Compression Method for Human Perception and Machine Vision Tasks](https://mlsanity.com/paper/2501.04329/) (2025)
- [Accuracy Improvement of Object Detection in VVC Coded Video Using YOLO-v7 Features](https://mlsanity.com/paper/2304.00689/) (2023)
- [Base Layer Efficiency in Scalable Human-Machine Coding](https://mlsanity.com/paper/2307.02430/) (2023)
- [Refining Coded Image in Human Vision Layer Using CNN-Based Post-Processing](https://mlsanity.com/paper/2405.11894/) (2024)
- [Towards Point Cloud Compression for Machine Perception: A Simple and Strong Baseline by Learning the Octree Depth Level Predictor](https://mlsanity.com/paper/2406.00791/) (2024)
- [A New Image Codec Paradigm for Human and Machine Uses](https://mlsanity.com/paper/2112.10071/) (2021)
- [VVC+M: Plug and Play Scalable Image Coding for Humans and Machines](https://mlsanity.com/paper/2305.10453/) (2023)
- [Learned Point Cloud Compression for Classification](https://mlsanity.com/paper/2308.05959/) (2023)
- [Bridging the gap between image coding for machines and humans](https://mlsanity.com/paper/2401.10732/) (2024)
- [Towards Task-Compatible Compressible Representations](https://mlsanity.com/paper/2405.10244/) (2024)
- [Explicit Residual-Based Scalable Image Coding for Humans and Machines](https://mlsanity.com/paper/2506.19297/) (2025)
- [SFU-HW-Tracks-v1: Object Tracking Dataset on Raw Video Sequences](https://mlsanity.com/paper/2112.14934/) (2021)

## Abstract (arXiv)

At present, and increasingly so in the future, much of the captured visual content will not be seen by humans. Instead, it will be used for automated machine vision analytics and may require occasional human viewing. Examples of such applications include traffic monitoring, visual surveillance, autonomous navigation, and industrial machine vision. To address such requirements, we develop an end-to-end learned image codec whose latent space is designed to support scalability from simpler to more complicated tasks. The simplest task is assigned to a subset of the latent space (the base layer), while more complicated tasks make use of additional subsets of the latent space, i.e., both the base and enhancement layer(s). For the experiments, we establish a 2-layer and a 3-layer model, each of which offers input reconstruction for human vision, plus machine vision task(s), and compare them with relevant benchmarks. The experiments show that our scalable codecs offer 37%-80% bitrate savings on machine vision tasks compared to best alternatives, while being comparable to state-of-the-art image codecs in terms of input reconstruction.