# End-to-End Rate-Distortion Optimization for Bi-Directional Learned Video Compression

arXiv: https://arxiv.org/abs/2008.05028 · HTML: https://mlsanity.com/paper/2008.05028/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Aug 2020.

## Built on by (most cited first, 8 of 8)

- [End-to-End Rate-Distortion Optimized Learned Hierarchical Bi-Directional Video Compression](https://mlsanity.com/paper/2112.09529/) (2021)
- [Bi-Directional Deep Contextual Video Compression](https://mlsanity.com/paper/2408.08604/) (2024)
- [IBVC: Interpolation-driven B-frame Video Compression](https://mlsanity.com/paper/2309.13835/) (2023)
- [Distilled Low Rank Neural Radiance Field with Quantization for Light Field Compression](https://mlsanity.com/paper/2208.00164/) (2022)
- [Neural B-frame Video Compression with Bi-directional Reference Harmonization](https://mlsanity.com/paper/2511.08938/) (2025)
- [DFPN: Deformable Frame Prediction Network](https://mlsanity.com/paper/2105.12794/) (2021)
- [Fine-Grained Motion Compression and Selective Temporal Fusion for Neural B-Frame Video Coding](https://mlsanity.com/paper/2506.07709/) (2025)
- [A Combined Deep Learning based End-to-End Video Coding Architecture for YUV Color Space](https://mlsanity.com/paper/2104.00807/) (2021)

## Abstract (arXiv)

Conventional video compression methods employ a linear transform and block motion model, and the steps of motion estimation, mode and quantization parameter selection, and entropy coding are optimized individually due to combinatorial nature of the end-to-end optimization problem. Learned video compression allows end-to-end rate-distortion optimized training of all nonlinear modules, quantization parameter and entropy model simultaneously. While previous work on learned video compression considered training a sequential video codec based on end-to-end optimization of cost averaged over pairs of successive frames, it is well-known in conventional video compression that hierarchical, bi-directional coding outperforms sequential compression. In this paper, we propose for the first time end-to-end optimization of a hierarchical, bi-directional motion compensated learned codec by accumulating cost function over fixed-size groups of pictures (GOP). Experimental results show that the rate-distortion performance of our proposed learned bi-directional it GOP coder outperforms the state-of-the-art end-to-end optimized learned sequential compression as expected.