# Fine-Grained Motion Compression and Selective Temporal Fusion for Neural B-Frame Video Coding

arXiv: https://arxiv.org/abs/2506.07709 · HTML: https://mlsanity.com/paper/2506.07709/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 85 references that are in the map.
- First posted on arXiv 9 Jun 2025.

## Builds on (most cited first, 30 of 49)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [Efficient Attention: Attention with Linear Complexities](https://mlsanity.com/paper/1812.01243/) (2018)
- [NeRV: Neural Representations for Videos](https://mlsanity.com/paper/2110.13903/) (2021)
- [Deep Contextual Video Compression](https://mlsanity.com/paper/2109.15047/) (2021)
- [Neural Video Compression with Diverse Contexts](https://mlsanity.com/paper/2302.14402/) (2023)
- [FVC: A New Framework towards Deep Video Compression in Feature Space](https://mlsanity.com/paper/2105.09600/) (2021)
- [Video Compression through Image Interpolation](https://mlsanity.com/paper/1804.06919/) (2018)
- [Hybrid Spatial-Temporal Entropy Modelling for Neural Video Compression](https://mlsanity.com/paper/2207.05894/) (2022)
- [Neural Video Compression with Feature Modulation](https://mlsanity.com/paper/2402.17414/) (2024)
- [Temporal Context Mining for Learned Video Compression](https://mlsanity.com/paper/2111.13850/) (2021)
- [M-LVC: Multiple Frames Prediction for Learned Video Compression](https://mlsanity.com/paper/2004.10290/) (2020)
- [Learning for Video Compression with Hierarchical Quality and Recurrent Enhancement](https://mlsanity.com/paper/2003.01966/) (2020)
- [Learned Video Compression](https://mlsanity.com/paper/1811.06981/) (2018)
- [VCT: A Video Compression Transformer](https://mlsanity.com/paper/2206.07307/) (2022)
- [Content Adaptive and Error Propagation Aware Deep Video Compression](https://mlsanity.com/paper/2003.11282/) (2020)
- [HiNeRV: Video Compression with Hierarchical Encoding-based Neural Representation](https://mlsanity.com/paper/2306.09818/) (2023)
- [ELF-VC: Efficient Learned Flexible-Rate Video Coding](https://mlsanity.com/paper/2104.14335/) (2021)
- [Learning for Video Compression with Recurrent Auto-Encoder and Recurrent Probability Model](https://mlsanity.com/paper/2006.13560/) (2020)
- [Improving Deep Video Compression by Resolution-adaptive Flow Coding](https://mlsanity.com/paper/2009.05982/) (2020)
- [Coarse-to-fine Deep Video Coding with Hyperprior-guided Mode Prediction](https://mlsanity.com/paper/2206.07460/) (2022)
- [CANF-VC: Conditional Augmented Normalizing Flows for Video Compression](https://mlsanity.com/paper/2207.05315/) (2022)
- [Conditional Entropy Coding for Efficient Video Compression](https://mlsanity.com/paper/2008.09180/) (2020)
- [Neural Video Coding using Multiscale Motion Compensation and Spatiotemporal Context Model](https://mlsanity.com/paper/2007.04574/) (2020)
- [Extending Neural P-frame Codecs for B-frame Coding](https://mlsanity.com/paper/2104.00531/) (2021)
- [End-to-End Rate-Distortion Optimized Learned Hierarchical Bi-Directional Video Compression](https://mlsanity.com/paper/2112.09529/) (2021)
- [Learning Image and Video Compression through Spatial-Temporal Energy Compaction](https://mlsanity.com/paper/1906.09683/) (2019)
- [Spatial Decomposition and Temporal Fusion based Inter Prediction for Learned Video Compression](https://mlsanity.com/paper/2401.15864/) (2024)
- [Bi-Directional Deep Contextual Video Compression](https://mlsanity.com/paper/2408.08604/) (2024)

## Built on by (most cited first, 2 of 2)

- [Neural Video Compression with Domain Transfer](https://mlsanity.com/paper/2605.13476/) (2026)
- [DCVC-MV: Deep Contextual Multiview Video Compression with Efficient Inter-View Prediction](https://mlsanity.com/paper/2509.03922/) (2025)

## Abstract (arXiv)

With the remarkable progress in neural P-frame video coding, neural B-frame coding has recently emerged as a critical research direction. However, most existing neural B-frame codecs directly adopt P-frame coding tools without adequately addressing the unique challenges of B-frame compression, leading to suboptimal performance. To bridge this gap, we propose novel enhancements for motion compression and temporal fusion for neural B-frame coding. First, we design a fine-grained motion compression method. This method incorporates an interactive dual-branch motion auto-encoder with per-branch adaptive quantization steps, which enables fine-grained compression of bi-directional motion vectors while accommodating their asymmetric bitrate allocation and reconstruction quality requirements. Furthermore, this method involves an interactive motion entropy model that exploits correlations between bi-directional motion latent representations by interactively leveraging partitioned latent segments as directional priors. Second, we propose a selective temporal fusion method that predicts bi-directional fusion weights to achieve discriminative utilization of bi-directional multi-scale temporal contexts with varying qualities. Additionally, this method introduces a hyperprior-based implicit alignment mechanism for contextual entropy modeling. By treating the hyperprior as a surrogate for the contextual latent representation, this mechanism implicitly mitigates the misalignment in the fused bi-directional temporal priors. Extensive experiments demonstrate that our proposed codec achieves an average BD-rate reduction of approximately 10% compared to the state-of-the-art neural B-frame codec, DCVC-B, and delivers comparable or even superior compression performance to the H.266/VVC reference software under random-access configurations.