# Learned Video Compression via Joint Spatial-Temporal Correlation Exploration

arXiv: https://arxiv.org/abs/1912.06348 · HTML: https://mlsanity.com/paper/1912.06348/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Dec 2019.

## Built on by (most cited first, 28 of 28)

- [Hybrid Spatial-Temporal Entropy Modelling for Neural Video Compression](https://mlsanity.com/paper/2207.05894/) (2022)
- [Temporal Context Mining for Learned Video Compression](https://mlsanity.com/paper/2111.13850/) (2021)
- [Implicit Neural Video Compression](https://mlsanity.com/paper/2112.11312/) (2021)
- [Perceptual Learned Video Compression with Recurrent Conditional GAN](https://mlsanity.com/paper/2109.03082/) (2021)
- [Overfitting for Fun and Profit: Instance-Adaptive Data Compression](https://mlsanity.com/paper/2101.08687/) (2021)
- [Extending Neural P-frame Codecs for B-frame Coding](https://mlsanity.com/paper/2104.00531/) (2021)
- [Spatial Decomposition and Temporal Fusion based Inter Prediction for Learned Video Compression](https://mlsanity.com/paper/2401.15864/) (2024)
- [Bi-Directional Deep Contextual Video Compression](https://mlsanity.com/paper/2408.08604/) (2024)
- [MobileCodec: Neural Inter-frame Video Compression on Mobile Devices](https://mlsanity.com/paper/2207.08338/) (2022)
- [Instance-Adaptive Video Compression: Improving Neural Codecs by Training on the Test Set](https://mlsanity.com/paper/2111.10302/) (2021)
- [Advancing Learned Video Compression with In-loop Frame Prediction](https://mlsanity.com/paper/2211.07004/) (2022)
- [End-to-end Neural Video Coding Using a Compound Spatiotemporal Representation](https://mlsanity.com/paper/2108.04103/) (2021)
- [Insights from Generative Modeling for Neural Video Compression](https://mlsanity.com/paper/2107.13136/) (2021)
- [Learned Video Compression via Heterogeneous Deformable Compensation Network](https://mlsanity.com/paper/2207.04589/) (2022)
- [BiECVC: Gated Diversification of Bidirectional Contexts for Learned Video Compression](https://mlsanity.com/paper/2505.09193/) (2025)
- [Semantically Video Coding: Instill Static-Dynamic Clues into Structured Bitstream for AI Tasks](https://mlsanity.com/paper/2201.10162/) (2022)
- [AccMPEG: Optimizing Video Encoding for Video Analytics](https://mlsanity.com/paper/2204.12534/) (2022)
- [Prediction and Reference Quality Adaptation for Learned Video Compression](https://mlsanity.com/paper/2406.14118/) (2024)
- [Spatiotemporal Entropy Model is All You Need for Learned Video Compression](https://mlsanity.com/paper/2104.06083/) (2021)
- [ERA: Entity Relationship Aware Video Summarization with Wasserstein GAN](https://mlsanity.com/paper/2109.02625/) (2021)
- [Exploring Long- and Short-Range Temporal Information for Learned Video Compression](https://mlsanity.com/paper/2208.03754/) (2022)
- [NVC-1B: A Large Neural Video Coding Model](https://mlsanity.com/paper/2407.19402/) (2024)
- [High-Efficiency Neural Video Compression via Hierarchical Predictive Learning](https://mlsanity.com/paper/2410.02598/) (2024)
- [Motion Free B-frame Coding for Neural Video Compression](https://mlsanity.com/paper/2411.17160/) (2024)
- [Viewport-based Neural 360deg Image Compression](https://mlsanity.com/paper/2603.22776/) (2026)
- [Neural Weight Step Video Compression](https://mlsanity.com/paper/2112.01504/) (2021)
- [MNeRV: A Multilayer Neural Representation for Videos](https://mlsanity.com/paper/2407.07347/) (2024)
- [A Causal Diffusion Model for Video Reconstruction from Ultra-Low-Bitrate Representations](https://mlsanity.com/paper/2602.13837/) (2026)

## Abstract (arXiv)

Traditional video compression technologies have been developed over decades in pursuit of higher coding efficiency. Efficient temporal information representation plays a key role in video coding. Thus, in this paper, we propose to exploit the temporal correlation using both first-order optical flow and second-order flow prediction. We suggest an one-stage learning approach to encapsulate flow as quantized features from consecutive frames which is then entropy coded with adaptive contexts conditioned on joint spatial-temporal priors to exploit second-order correlations. Joint priors are embedded in autoregressive spatial neighbors, co-located hyper elements and temporal neighbors using ConvLSTM recurrently. We evaluate our approach for the low-delay scenario with High-Efficiency Video Coding (H.265/HEVC), H.264/AVC and another learned video compression method, following the common test settings. Our work offers the state-of-the-art performance, with consistent gains across all popular test sequences.