# Learned Video Compression via Heterogeneous Deformable Compensation Network

arXiv: https://arxiv.org/abs/2207.04589 · HTML: https://mlsanity.com/paper/2207.04589/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 11 Jul 2022.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [DVC: An End-to-end Deep Video Compression Framework](https://mlsanity.com/paper/1812.00101/) (2018)
- [CompressAI: a PyTorch library and evaluation platform for end-to-end compression research](https://mlsanity.com/paper/2011.03029/) (2020)
- [TDAN: Temporally Deformable Alignment Network for Video Super-Resolution](https://mlsanity.com/paper/1812.02898/) (2018)
- [Deep Contextual Video Compression](https://mlsanity.com/paper/2109.15047/) (2021)
- [FVC: A New Framework towards Deep Video Compression in Feature Space](https://mlsanity.com/paper/2105.09600/) (2021)
- [Video Compression through Image Interpolation](https://mlsanity.com/paper/1804.06919/) (2018)
- [Context-adaptive Entropy Model for End-to-end Optimized Image Compression](https://mlsanity.com/paper/1809.10452/) (2018)
- [Density Modeling of Images using a Generalized Normalization Transformation](https://mlsanity.com/paper/1511.06281/) (2015)
- [Video Compression With Rate-Distortion Autoencoders](https://mlsanity.com/paper/1908.05717/) (2019)
- [Learning for Video Compression with Hierarchical Quality and Recurrent Enhancement](https://mlsanity.com/paper/2003.01966/) (2020)
- [Learned Video Compression](https://mlsanity.com/paper/1811.06981/) (2018)
- [Content Adaptive and Error Propagation Aware Deep Video Compression](https://mlsanity.com/paper/2003.11282/) (2020)
- [Learning for Video Compression with Recurrent Auto-Encoder and Recurrent Probability Model](https://mlsanity.com/paper/2006.13560/) (2020)
- [Improving Deep Video Compression by Resolution-adaptive Flow Coding](https://mlsanity.com/paper/2009.05982/) (2020)
- [Learned Video Compression via Joint Spatial-Temporal Correlation Exploration](https://mlsanity.com/paper/1912.06348/) (2019)
- [HetConv: Heterogeneous Kernel-Based Convolutions for Deep CNNs](https://mlsanity.com/paper/1903.04120/) (2019)
- [End-to-end Neural Video Coding Using a Compound Spatiotemporal Representation](https://mlsanity.com/paper/2108.04103/) (2021)
- [Learned Multi-Resolution Variable-Rate Image Compression with Octave-based Residual Blocks](https://mlsanity.com/paper/2012.15463/) (2020)
- [Light Field Image Coding Using VVC standard and View Synthesis based on Dual Discriminator GAN](https://mlsanity.com/paper/2103.04201/) (2021)

## Built on by (most cited first, 8 of 8)

- [Bi-Directional Deep Contextual Video Compression](https://mlsanity.com/paper/2408.08604/) (2024)
- [IBVC: Interpolation-driven B-frame Video Compression](https://mlsanity.com/paper/2309.13835/) (2023)
- [Content-Aware Mamba for Learned Image Compression](https://mlsanity.com/paper/2508.02192/) (2025)
- [Exploring Long- and Short-Range Temporal Information for Learned Video Compression](https://mlsanity.com/paper/2208.03754/) (2022)
- [Fine-Grained Motion Compression and Selective Temporal Fusion for Neural B-Frame Video Coding](https://mlsanity.com/paper/2506.07709/) (2025)
- [Adaptive Learned Image Compression with Graph Neural Networks](https://mlsanity.com/paper/2603.25316/) (2026)
- [KD-NVC: A Search-and-Distill Framework to Accelerate Neural Video Coding](https://mlsanity.com/paper/2606.04595/) (2026)
- [ENCORE: Event-Assisted Complementary Motion Refinement for Learned Video Compression](https://mlsanity.com/paper/2607.28020/) (2026)

## Abstract (arXiv)

Learned video compression has recently emerged as an essential research topic in developing advanced video compression technologies, where motion compensation is considered one of the most challenging issues. In this paper, we propose a learned video compression framework via heterogeneous deformable compensation strategy (HDCVC) to tackle the problems of unstable compression performance caused by single-size deformable kernels in downsampled feature domain. More specifically, instead of utilizing optical flow warping or single-size-kernel deformable alignment, the proposed algorithm extracts features from the two adjacent frames to estimate content-adaptive heterogeneous deformable (HetDeform) kernel offsets. Then we transform the reference features with the HetDeform convolution to accomplish motion compensation. Moreover, we design a Spatial-Neighborhood-Conditioned Divisive Normalization (SNCDN) to achieve more effective data Gaussianization combined with the Generalized Divisive Normalization. Furthermore, we propose a multi-frame enhanced reconstruction module for exploiting context and temporal information for final quality enhancement. Experimental results indicate that HDCVC achieves superior performance than the recent state-of-the-art learned video compression approaches.