# Flexible-Rate Learned Hierarchical Bi-Directional Video Compression With Motion Refinement and Frame-Level Bit Allocation

arXiv: https://arxiv.org/abs/2206.13613 · HTML: https://mlsanity.com/paper/2206.13613/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 22 references that are in the map.
- First posted on arXiv 27 Jun 2022.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [DVC: An End-to-end Deep Video Compression Framework](https://mlsanity.com/paper/1812.00101/) (2018)
- [CompressAI: a PyTorch library and evaluation platform for end-to-end compression research](https://mlsanity.com/paper/2011.03029/) (2020)
- [Channel-wise Autoregressive Entropy Models for Learned Image Compression](https://mlsanity.com/paper/2007.08739/) (2020)
- [Learning for Video Compression with Hierarchical Quality and Recurrent Enhancement](https://mlsanity.com/paper/2003.01966/) (2020)
- [Asymmetric Gained Deep Image Compression With Continuous Rate Adaptation](https://mlsanity.com/paper/2003.02012/) (2020)
- [ELF-VC: Efficient Learned Flexible-Rate Video Coding](https://mlsanity.com/paper/2104.14335/) (2021)
- [Learning for Video Compression with Recurrent Auto-Encoder and Recurrent Probability Model](https://mlsanity.com/paper/2006.13560/) (2020)
- [End-to-End Rate-Distortion Optimized Learned Hierarchical Bi-Directional Video Compression](https://mlsanity.com/paper/2112.09529/) (2021)
- [Conditional Coding for Flexible Learned Video Compression](https://mlsanity.com/paper/2104.07930/) (2021)

## Built on by (most cited first, 7 of 7)

- [Hierarchical B-frame Video Coding Using Two-Layer CANF without Motion Coding](https://mlsanity.com/paper/2304.02690/) (2023)
- [IBVC: Interpolation-driven B-frame Video Compression](https://mlsanity.com/paper/2309.13835/) (2023)
- [Bit Allocation using Optimization](https://mlsanity.com/paper/2209.09422/) (2022)
- [Multi-Scale Deformable Alignment and Content-Adaptive Inference for Flexible-Rate Bi-Directional Video Compression](https://mlsanity.com/paper/2306.16544/) (2023)
- [Neural Video Compression with Temporal Layer-Adaptive Hierarchical B-frame Coding](https://mlsanity.com/paper/2308.15791/) (2023)
- [Learned Rate Control for Frame-Level Adaptive Neural Video Compression via Dynamic Neural Network](https://mlsanity.com/paper/2508.20709/) (2025)
- [Correcting the Sub-optimal Bit Allocation](https://mlsanity.com/paper/2209.14575/) (2022)

## Abstract (arXiv)

This paper presents improvements and novel additions to our recent work on end-to-end optimized hierarchical bi-directional video compression to further advance the state-of-the-art in learned video compression. As an improvement, we combine motion estimation and prediction modules and compress refined residual motion vectors for improved rate-distortion performance. As novel addition, we adapted the gain unit proposed for image compression to flexible-rate video compression in two ways: first, the gain unit enables a single encoder model to operate at multiple rate-distortion operating points; second, we exploit the gain unit to control bit allocation among intra-coded vs. bi-directionally coded frames by fine tuning corresponding models for truly flexible-rate learned video coding. Experimental results demonstrate that we obtain state-of-the-art rate-distortion performance exceeding those of all prior art in learned video coding.