# DVC: An End-to-end Deep Video Compression Framework

arXiv: https://arxiv.org/abs/1812.00101 · HTML: https://mlsanity.com/paper/1812.00101/

## Key facts

- Cited by 332 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Nov 2018.

## Built on by (most cited first, 30 of 332)

- [Finite Scalar Quantization: VQ-VAE Made Simple](https://mlsanity.com/paper/2309.15505/) (2023)
- [NeRV: Neural Representations for Videos](https://mlsanity.com/paper/2110.13903/) (2021)
- [Diffusion Probabilistic Modeling for Video Generation](https://mlsanity.com/paper/2203.09481/) (2022)
- [Deep Contextual Video Compression](https://mlsanity.com/paper/2109.15047/) (2021)
- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [Neural Video Compression with Diverse Contexts](https://mlsanity.com/paper/2302.14402/) (2023)
- [FVC: A New Framework towards Deep Video Compression in Feature Space](https://mlsanity.com/paper/2105.09600/) (2021)
- [Hybrid Spatial-Temporal Entropy Modelling for Neural Video Compression](https://mlsanity.com/paper/2207.05894/) (2022)
- [Neural Video Compression with Feature Modulation](https://mlsanity.com/paper/2402.17414/) (2024)
- [Image and Video Tokenization with Binary Spherical Quantization](https://mlsanity.com/paper/2406.07548/) (2024)
- [HNeRV: A Hybrid Neural Representation for Videos](https://mlsanity.com/paper/2304.02633/) (2023)
- [VideoControlNet: A Motion-Guided Video-to-Video Translation Framework by Using Diffusion Model with ControlNet](https://mlsanity.com/paper/2307.14073/) (2023)
- [COIN++: Neural Compression Across Modalities](https://mlsanity.com/paper/2201.12904/) (2022)
- [Towards Practical Real-Time Neural Video Compression](https://mlsanity.com/paper/2502.20762/) (2025)
- [An Introduction to Neural Data Compression](https://mlsanity.com/paper/2202.06533/) (2022)
- [VCT: A Video Compression Transformer](https://mlsanity.com/paper/2206.07307/) (2022)
- [HiNeRV: Video Compression with Hierarchical Encoding-based Neural Representation](https://mlsanity.com/paper/2306.09818/) (2023)
- [ELF-VC: Efficient Learned Flexible-Rate Video Coding](https://mlsanity.com/paper/2104.14335/) (2021)
- [VoxelContext-Net: An Octree based Framework for Point Cloud Compression](https://mlsanity.com/paper/2105.02158/) (2021)
- [Dense Optical Tracking: Connecting the Dots](https://mlsanity.com/paper/2312.00786/) (2023)
- [Coarse-to-fine Deep Video Coding with Hyperprior-guided Mode Prediction](https://mlsanity.com/paper/2206.07460/) (2022)
- [Implicit Neural Video Compression](https://mlsanity.com/paper/2112.11312/) (2021)
- [Wireless Deep Video Semantic Transmission](https://mlsanity.com/paper/2205.13129/) (2022)
- [CANF-VC: Conditional Augmented Normalizing Flows for Video Compression](https://mlsanity.com/paper/2207.05315/) (2022)
- [FFNeRV: Flow-Guided Frame-Wise Neural Representations for Videos](https://mlsanity.com/paper/2212.12294/) (2022)
- [MD-VQA: Multi-Dimensional Quality Assessment for UGC Live Videos](https://mlsanity.com/paper/2303.14933/) (2023)
- [DeepWiVe: Deep-Learning-Aided Wireless Video Transmission](https://mlsanity.com/paper/2111.13034/) (2021)
- [DNeRV: Modeling Inherent Dynamics via Difference Neural Representation for Videos](https://mlsanity.com/paper/2304.06544/) (2023)
- [Perceptual Learned Video Compression with Recurrent Conditional GAN](https://mlsanity.com/paper/2109.03082/) (2021)
- [Overfitting for Fun and Profit: Instance-Adaptive Data Compression](https://mlsanity.com/paper/2101.08687/) (2021)

## Abstract (arXiv)

Conventional video compression approaches use the predictive coding architecture and encode the corresponding motion information and residual information. In this paper, taking advantage of both classical architecture in the conventional video compression method and the powerful non-linear representation ability of neural networks, we propose the first end-to-end video compression deep model that jointly optimizes all the components for video compression. Specifically, learning based optical flow estimation is utilized to obtain the motion information and reconstruct the current frames. Then we employ two auto-encoder style neural networks to compress the corresponding motion and residual information. All the modules are jointly learned through a single loss function, in which they collaborate with each other by considering the trade-off between reducing the number of compression bits and improving quality of the decoded video. Experimental results show that the proposed approach can outperform the widely used video coding standard H.264 in terms of PSNR and be even on par with the latest standard H.265 in terms of MS-SSIM. Code is released at https://github.com/GuoLusjtu/DVC.