# An Emerging Coding Paradigm VCM: A Scalable Coding Approach Beyond Feature and Signal

arXiv: https://arxiv.org/abs/2001.03004 · HTML: https://mlsanity.com/paper/2001.03004/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jan 2020.

## Built on by (most cited first, 8 of 8)

- [Image Coding for Machines with Omnipotent Feature Learning](https://mlsanity.com/paper/2207.01932/) (2022)
- [VNVC: A Versatile Neural Video Coding Framework for Efficient Human-Machine Vision](https://mlsanity.com/paper/2306.10681/) (2023)
- [End-to-end Compression Towards Machine Vision: Network Architecture Design and Optimization](https://mlsanity.com/paper/2107.00328/) (2021)
- [Video Coding for Machine: Compact Visual Representation Compression for Intelligent Collaborative Analytics](https://mlsanity.com/paper/2110.09241/) (2021)
- [AccMPEG: Optimizing Video Encoding for Video Analytics](https://mlsanity.com/paper/2204.12534/) (2022)
- [Task Oriented Video Coding: A Survey](https://mlsanity.com/paper/2208.07313/) (2022)
- [Deep Video Codec Control for Vision Models](https://mlsanity.com/paper/2308.16215/) (2023)
- [Revisit Visual Representation in Analytics Taxonomy: A Compression Perspective](https://mlsanity.com/paper/2106.08512/) (2021)

## Abstract (arXiv)

In this paper, we study a new problem arising from the emerging MPEG standardization effort Video Coding for Machine (VCM), which aims to bridge the gap between visual feature compression and classical video coding. VCM is committed to address the requirement of compact signal representation for both machine and human vision in a more or less scalable way. To this end, we make endeavors in leveraging the strength of predictive and generative models to support advanced compression techniques for both machine and human vision tasks simultaneously, in which visual features serve as a bridge to connect signal-level and task-level compact representations in a scalable manner. Specifically, we employ a conditional deep generation network to reconstruct video frames with the guidance of learned motion pattern. By learning to extract sparse motion pattern via a predictive model, the network elegantly leverages the feature representation to generate the appearance of to-be-coded frames via a generative model, relying on the appearance of the coded key frames. Meanwhile, the sparse motion pattern is compact and highly effective for high-level vision tasks, e.g. action recognition. Experimental results demonstrate that our method yields much better reconstruction quality compared with the traditional video codecs (0.0063 gain in SSIM), as well as state-of-the-art action recognition performance over highly compressed videos (9.4% gain in recognition accuracy), which showcases a promising paradigm of coding signal for both human and machine vision.