# Towards Image Understanding from Deep Compression without Decoding

arXiv: https://arxiv.org/abs/1803.06131 · HTML: https://mlsanity.com/paper/1803.06131/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Mar 2018.

## Built on by (most cited first, 30 of 52)

- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [Improving Statistical Fidelity for Neural Image Compression with Implicit Local Likelihood Models](https://mlsanity.com/paper/2301.11189/) (2023)
- [An Introduction to Neural Data Compression](https://mlsanity.com/paper/2202.06533/) (2022)
- [Towards Semantic Communications: Deep Learning-Based Image Semantic Coding](https://mlsanity.com/paper/2208.04094/) (2022)
- [TCGL: Temporal Contrastive Graph for Self-supervised Video Representation Learning](https://mlsanity.com/paper/2112.03587/) (2021)
- [Towards End-to-End Image Compression and Analysis with Transformers](https://mlsanity.com/paper/2112.09300/) (2021)
- [Towards Discriminative Representation Learning for Unsupervised Person Re-identification](https://mlsanity.com/paper/2108.03439/) (2021)
- [Preprocessing Enhanced Image Compression for Machine Vision](https://mlsanity.com/paper/2206.05650/) (2022)
- [DuetFace: Collaborative Privacy-Preserving Face Recognition via Channel Splitting in the Frequency Domain](https://mlsanity.com/paper/2207.07340/) (2022)
- [PerCo (SD): Open Perceptual Compression](https://mlsanity.com/paper/2409.20255/) (2024)
- [Learned Image Compression for Machine Perception](https://mlsanity.com/paper/2111.02249/) (2021)
- [Learning-Driven Lossy Image Compression; A Comprehensive Survey](https://mlsanity.com/paper/2201.09240/) (2022)
- [Generative Visual Compression: A Review](https://mlsanity.com/paper/2402.02140/) (2024)
- [End-to-end Compression Towards Machine Vision: Network Architecture Design and Optimization](https://mlsanity.com/paper/2107.00328/) (2021)
- [FreqNet: A Frequency-domain Image Super-Resolution Network with Dicrete Cosine Transform](https://mlsanity.com/paper/2111.10800/) (2021)
- [Decentralized Low-Latency Collaborative Inference via Ensembles on the Edge](https://mlsanity.com/paper/2206.03165/) (2022)
- [Semantic Segmentation in Learned Compressed Domain](https://mlsanity.com/paper/2209.01355/) (2022)
- [Semantically Video Coding: Instill Static-Dynamic Clues into Structured Bitstream for AI Tasks](https://mlsanity.com/paper/2201.10162/) (2022)
- [Perceptual Video Coding for Machines via Satisfied Machine Ratio Modeling](https://mlsanity.com/paper/2211.06797/) (2022)
- [NeuCo-Bench: A Novel Benchmark Framework for Neural Embeddings in Earth Observation](https://mlsanity.com/paper/2510.17914/) (2025)
- [Pseudocylindrical Convolutions for Learned Omnidirectional Image Compression](https://mlsanity.com/paper/2112.13227/) (2021)
- [Machine Perception-Driven Image Compression: A Layered Generative Approach](https://mlsanity.com/paper/2304.06896/) (2023)
- [Task-Aware Encoder Control for Deep Video Compression](https://mlsanity.com/paper/2404.04848/) (2024)
- [Does lossy image compression affect racial bias within face recognition?](https://mlsanity.com/paper/2208.07613/) (2022)
- [An Efficient Adaptive Compression Method for Human Perception and Machine Vision Tasks](https://mlsanity.com/paper/2501.04329/) (2025)
- [Identity Preserving Loss for Learned Image Compression](https://mlsanity.com/paper/2204.10869/) (2022)
- [Towards Point Cloud Compression for Machine Perception: A Simple and Strong Baseline by Learning the Octree Depth Level Predictor](https://mlsanity.com/paper/2406.00791/) (2024)
- [Distributed Semantic Segmentation with Efficient Joint Source and Task Decoding](https://mlsanity.com/paper/2407.11224/) (2024)
- [Unified Coding for Both Human Perception and Generalized Machine Analytics with CLIP Supervision](https://mlsanity.com/paper/2501.04579/) (2025)
- [A New Image Codec Paradigm for Human and Machine Uses](https://mlsanity.com/paper/2112.10071/) (2021)

## Abstract (arXiv)

Motivated by recent work on deep neural network (DNN)-based image compression methods showing potential improvements in image quality, savings in storage, and bandwidth reduction, we propose to perform image understanding tasks such as classification and segmentation directly on the compressed representations produced by these compression methods. Since the encoders and decoders in DNN-based compression methods are neural networks with feature-maps as internal representations of the images, we directly integrate these with architectures for image understanding. This bypasses decoding of the compressed representation into RGB space and reduces computational cost. Our study shows that accuracies comparable to networks that operate on compressed RGB images can be achieved while reducing the computational complexity up to 2×. Furthermore, we show that synergies are obtained by jointly training compression networks with classification networks on the compressed representations, improving image quality, classification accuracy, and segmentation performance. We find that inference from compressed representations is particularly advantageous compared to inference from compressed RGB images for aggressive compression rates.