# Switchable Temporal Propagation Network

arXiv: https://arxiv.org/abs/1804.08758 · HTML: https://mlsanity.com/paper/1804.08758/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2018.

## Built on by (most cited first, 9 of 9)

- [Unsupervised Visual Representation Learning by Synchronous Momentum Grouping](https://mlsanity.com/paper/2207.06167/) (2022)
- [Deep Video Prior for Video Consistency and Propagation](https://mlsanity.com/paper/2201.11632/) (2022)
- [Deep Learning Technique for Human Parsing: A Survey and Outlook](https://mlsanity.com/paper/2301.00394/) (2023)
- [Video Colorization with Pre-trained Text-to-Image Diffusion Models](https://mlsanity.com/paper/2306.01732/) (2023)
- [Exemplar-based Video Colorization with Long-term Spatiotemporal Dependency](https://mlsanity.com/paper/2303.15081/) (2023)
- [DeepHS-HDRVideo: Deep High Speed High Dynamic Range Video Reconstruction](https://mlsanity.com/paper/2210.04429/) (2022)
- [Two Decades of Colorization and Decolorization for Images and Videos](https://mlsanity.com/paper/2204.13322/) (2022)
- [Temporal Consistent Automatic Video Colorization via Semantic Correspondence](https://mlsanity.com/paper/2305.07904/) (2023)
- [Computer-aided Colorization State-of-the-science: A Survey](https://mlsanity.com/paper/2410.02288/) (2024)

## Abstract (arXiv)

Videos contain highly redundant information between frames. Such redundancy has been extensively studied in video compression and encoding, but is less explored for more advanced video processing. In this paper, we propose a learnable unified framework for propagating a variety of visual properties of video images, including but not limited to color, high dynamic range (HDR), and segmentation information, where the properties are available for only a few key-frames. Our approach is based on a temporal propagation network (TPN), which models the transition-related affinity between a pair of frames in a purely data-driven manner. We theoretically prove two essential factors for TPN: (a) by regularizing the global transformation matrix as orthogonal, the "style energy" of the property can be well preserved during propagation; (b) such regularization can be achieved by the proposed switchable TPN with bi-directional training on pairs of frames. We apply the switchable TPN to three tasks: colorizing a gray-scale video based on a few color key-frames, generating an HDR video from a low dynamic range (LDR) video and a few HDR frames, and propagating a segmentation mask from the first frame in videos. Experimental results show that our approach is significantly more accurate and efficient than the state-of-the-art methods.