# Extending Neural P-frame Codecs for B-frame Coding

arXiv: https://arxiv.org/abs/2104.00531 · HTML: https://mlsanity.com/paper/2104.00531/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 48 references that are in the map.
- First posted on arXiv 30 Mar 2021.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [DVC: An End-to-end Deep Video Compression Framework](https://mlsanity.com/paper/1812.00101/) (2018)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Channel-wise Autoregressive Entropy Models for Learned Image Compression](https://mlsanity.com/paper/2007.08739/) (2020)
- [Full Resolution Image Compression with Recurrent Neural Networks](https://mlsanity.com/paper/1608.05148/) (2016)
- [Variable Rate Image Compression with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06085/) (2015)
- [Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations](https://mlsanity.com/paper/1704.00648/) (2017)
- [Quadratic video interpolation](https://mlsanity.com/paper/1911.00627/) (2019)
- [Video Compression through Image Interpolation](https://mlsanity.com/paper/1804.06919/) (2018)
- [Conditional Probability Models for Deep Image Compression](https://mlsanity.com/paper/1801.04260/) (2018)
- [Real-Time Adaptive Image Compression](https://mlsanity.com/paper/1705.05823/) (2017)
- [Density Modeling of Images using a Generalized Normalization Transformation](https://mlsanity.com/paper/1511.06281/) (2015)
- [Variable Rate Deep Image Compression With a Conditional Autoencoder](https://mlsanity.com/paper/1909.04802/) (2019)
- [M-LVC: Multiple Frames Prediction for Learned Video Compression](https://mlsanity.com/paper/2004.10290/) (2020)
- [Video Compression With Rate-Distortion Autoencoders](https://mlsanity.com/paper/1908.05717/) (2019)
- [Non-local Attention Optimized Deep Image Compression](https://mlsanity.com/paper/1904.09757/) (2019)
- [Learned Video Compression](https://mlsanity.com/paper/1811.06981/) (2018)
- [End-to-end optimization of nonlinear transform codes for perceptual quality](https://mlsanity.com/paper/1607.05006/) (2016)
- [Neural Video Coding using Multiscale Motion Compensation and Spatiotemporal Context Model](https://mlsanity.com/paper/2007.04574/) (2020)
- [Feedback Recurrent Autoencoder for Video Compression](https://mlsanity.com/paper/2004.04342/) (2020)
- [Learned Video Compression via Joint Spatial-Temporal Correlation Exploration](https://mlsanity.com/paper/1912.06348/) (2019)
- [Hierarchical Autoregressive Modeling for Neural Video Compression](https://mlsanity.com/paper/2010.10258/) (2020)
- [Learning Image and Video Compression through Spatial-Temporal Energy Compaction](https://mlsanity.com/paper/1906.09683/) (2019)
- [Progressive Neural Image Compression with Nested Quantization and Latent Ordering](https://mlsanity.com/paper/2102.02913/) (2021)
- [Deep Generative Video Compression](https://mlsanity.com/paper/1810.02845/) (2018)

## Built on by (most cited first, 30 of 30)

- [HiNeRV: Video Compression with Hierarchical Encoding-based Neural Representation](https://mlsanity.com/paper/2306.09818/) (2023)
- [Implicit Neural Video Compression](https://mlsanity.com/paper/2112.11312/) (2021)
- [AlphaVC: High-Performance and Efficient Learned Video Compression](https://mlsanity.com/paper/2207.14678/) (2022)
- [MobileCodec: Neural Inter-frame Video Compression on Mobile Devices](https://mlsanity.com/paper/2207.08338/) (2022)
- [Instance-Adaptive Video Compression: Improving Neural Codecs by Training on the Test Set](https://mlsanity.com/paper/2111.10302/) (2021)
- [B-CANF: Adaptive B-frame Coding with Conditional Augmented Normalizing Flows](https://mlsanity.com/paper/2209.01769/) (2022)
- [Hierarchical B-frame Video Coding Using Two-Layer CANF without Motion Coding](https://mlsanity.com/paper/2304.02690/) (2023)
- [Learning Cross-Scale Weighted Prediction for Efficient Neural Video Compression](https://mlsanity.com/paper/2112.13309/) (2021)
- [Neighbor Correspondence Matching for Flow-based Video Frame Synthesis](https://mlsanity.com/paper/2207.06763/) (2022)
- [IBVC: Interpolation-driven B-frame Video Compression](https://mlsanity.com/paper/2309.13835/) (2023)
- [High Visual-Fidelity Learned Video Compression](https://mlsanity.com/paper/2310.04679/) (2023)
- [BiECVC: Gated Diversification of Bidirectional Contexts for Learned Video Compression](https://mlsanity.com/paper/2505.09193/) (2025)
- [Versatile Learned Video Compression](https://mlsanity.com/paper/2111.03386/) (2021)
- [JNMR: Joint Non-linear Motion Regression for Video Frame Interpolation](https://mlsanity.com/paper/2206.04231/) (2022)
- [Boosting neural video codecs by exploiting hierarchical redundancy](https://mlsanity.com/paper/2208.04303/) (2022)
- [Semantic-Aware Adaptive Video Streaming Using Latent Diffusion Models for Wireless Networks](https://mlsanity.com/paper/2502.05695/) (2025)
- [Non-linear Motion Estimation for Video Frame Interpolation using Space-time Convolutions](https://mlsanity.com/paper/2201.11407/) (2022)
- [NU-Class Net: A Novel Approach for Video Quality Enhancement](https://mlsanity.com/paper/2401.01163/) (2024)
- [L-LBVC: Long-Term Motion Estimation and Prediction for Learned Bi-Directional Video Compression](https://mlsanity.com/paper/2504.02560/) (2025)
- [Region-of-Interest Based Neural Video Compression](https://mlsanity.com/paper/2203.01978/) (2022)
- [Reducing The Mismatch Between Marginal and Learned Distributions in Neural Video Compression](https://mlsanity.com/paper/2210.06596/) (2022)
- [Neural Video Compression with Temporal Layer-Adaptive Hierarchical B-frame Coding](https://mlsanity.com/paper/2308.15791/) (2023)
- [Low-Latency Neural Stereo Streaming](https://mlsanity.com/paper/2403.17879/) (2024)
- [Survey on Visual Signal Coding and Processing with Generative Models: Technologies, Standards and Optimization](https://mlsanity.com/paper/2405.14221/) (2024)
- [Fine-Grained Motion Compression and Selective Temporal Fusion for Neural B-Frame Video Coding](https://mlsanity.com/paper/2506.07709/) (2025)
- [Motion Free B-frame Coding for Neural Video Compression](https://mlsanity.com/paper/2411.17160/) (2024)
- [Learned Hierarchical B-frame Coding with Adaptive Feature Modulation for YUV 4:2:0 Content](https://mlsanity.com/paper/2212.14187/) (2022)
- [Neural-based Video Compression on Solar Dynamics Observatory Images](https://mlsanity.com/paper/2407.15730/) (2024)
- [Exploiting Latent Properties to Optimize Neural Codecs](https://mlsanity.com/paper/2501.01231/) (2025)
- [Neural B-Frame Coding: Tackling Domain Shift Issues with Lightweight Online Motion Resolution Adaptation](https://mlsanity.com/paper/2511.18724/) (2025)

## Abstract (arXiv)

While most neural video codecs address P-frame coding (predicting each frame from past ones), in this paper we address B-frame compression (predicting frames using both past and future reference frames). Our B-frame solution is based on the existing P-frame methods. As a result, B-frame coding capability can easily be added to an existing neural codec. The basic idea of our B-frame coding method is to interpolate the two reference frames to generate a single reference frame and then use it together with an existing P-frame codec to encode the input B-frame. Our studies show that the interpolated frame is a much better reference for the P-frame codec compared to using the previous frame as is usually done. Our results show that using the proposed method with an existing P-frame codec can lead to 28.5%saving in bit-rate on the UVG dataset compared to the P-frame codec while generating the same video quality.