# Task Oriented Video Coding: A Survey

arXiv: https://arxiv.org/abs/2208.07313 · HTML: https://mlsanity.com/paper/2208.07313/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 82 references that are in the map.
- First posted on arXiv 15 Aug 2022.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Video Coding for Machines: A Paradigm of Collaborative Compression and Intelligent Analytics](https://mlsanity.com/paper/2001.03569/) (2020)
- [Towards Coding for Human and Machine Vision: A Scalable Image Coding Approach](https://mlsanity.com/paper/2001.02915/) (2020)
- [Task-Oriented Image Transmission for Scene Classification in Unmanned Aerial Systems](https://mlsanity.com/paper/2112.10948/) (2021)
- [Task-driven Semantic Coding via Reinforcement Learning](https://mlsanity.com/paper/2106.03511/) (2021)
- [DeepN-JPEG: A Deep Neural Network Favorable JPEG-based Image Compression Framework](https://mlsanity.com/paper/1803.05788/) (2018)
- [Adversarial Crowdsourcing Through Robust Rank-One Matrix Completion](https://mlsanity.com/paper/2010.12181/) (2020)
- [Compact Descriptors for Video Analysis: the Emerging MPEG Standard](https://mlsanity.com/paper/1704.08141/) (2017)
- [Recent Standard Development Activities on Video Coding for Machines](https://mlsanity.com/paper/2105.12653/) (2021)
- [Video Coding for Machine: Compact Visual Representation Compression for Intelligent Collaborative Analytics](https://mlsanity.com/paper/2110.09241/) (2021)
- [Semi-supervised Domain Adaptive Structure Learning](https://mlsanity.com/paper/2112.06161/) (2021)
- [Scalable Video Coding for Humans and Machines](https://mlsanity.com/paper/2208.02512/) (2022)
- [Learning based Facial Image Compression with Semantic Fidelity Metric](https://mlsanity.com/paper/1812.10067/) (2018)
- [An Emerging Coding Paradigm VCM: A Scalable Coding Approach Beyond Feature and Signal](https://mlsanity.com/paper/2001.03004/) (2020)
- [Video Coding for Machines with Feature-Based Rate-Distortion Optimization](https://mlsanity.com/paper/2203.05890/) (2022)
- [Optimal Lockdown for Pandemic Control](https://mlsanity.com/paper/2010.12923/) (2020)
- [Semantically Video Coding: Instill Static-Dynamic Clues into Structured Bitstream for AI Tasks](https://mlsanity.com/paper/2201.10162/) (2022)
- [Finite-Sample Analysis of Decentralized Q-Learning for Stochastic Games](https://mlsanity.com/paper/2112.07859/) (2021)
- [Human-Machine Collaborative Video Coding Through Cuboidal Partitioning](https://mlsanity.com/paper/2102.01307/) (2021)
- [On Intra Video Coding and In-loop Filtering for Neural Object Detection Networks](https://mlsanity.com/paper/2203.05927/) (2022)
- [Optimal Vaccine Allocation for Pandemic Stabilization](https://mlsanity.com/paper/2109.04612/) (2021)
- [Video Coding for Machines: Partial transmission of SIFT features](https://mlsanity.com/paper/2201.02689/) (2022)

## Built on by (most cited first, 3 of 3)

- [Deep Video Codec Control for Vision Models](https://mlsanity.com/paper/2308.16215/) (2023)
- [Differentially Encoded Observation Spaces for Perceptive Reinforcement Learning](https://mlsanity.com/paper/2310.01767/) (2023)
- [PAT-VCM: Plug-and-Play Auxiliary Tokens for Video Coding for Machines](https://mlsanity.com/paper/2604.13294/) (2026)

## Abstract (arXiv)

Video coding technology has been continuously improved for higher compression ratio with higher resolution. However, the state-of-the-art video coding standards, such as H.265/HEVC and Versatile Video Coding, are still designed with the assumption the compressed video will be watched by humans. With the tremendous advance and maturation of deep neural networks in solving computer vision tasks, more and more videos are directly analyzed by deep neural networks without humans' involvement. Such a conventional design for video coding standard is not optimal when the compressed video is used by computer vision applications. While the human visual system is consistently sensitive to the content with high contrast, the impact of pixels on computer vision algorithms is driven by specific computer vision tasks. In this paper, we explore and summarize recent progress on computer vision task oriented video coding and emerging video coding standard, Video Coding for Machines.