# Fully Automatic Video Colorization with Self-Regularization and Diversity

arXiv: https://arxiv.org/abs/1908.01311 · HTML: https://mlsanity.com/paper/1908.01311/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Aug 2019.

## Built on by (most cited first, 30 of 40)

- [Test-Time Training on Video Streams](https://mlsanity.com/paper/2307.05014/) (2023)
- [Bringing Old Films Back to Life](https://mlsanity.com/paper/2203.17276/) (2022)
- [Blind Video Deflickering by Neural Filtering with a Flawed Atlas](https://mlsanity.com/paper/2303.08120/) (2023)
- [Video Demoireing with Relation-Based Temporal Consistency](https://mlsanity.com/paper/2204.02957/) (2022)
- [Temporally Consistent Video Colorization with Deep Feature Propagation and Self-regularization Learning](https://mlsanity.com/paper/2110.04562/) (2021)
- [VIDiff: Translating Videos via Multi-Modal Instructions with Diffusion Models](https://mlsanity.com/paper/2311.18837/) (2023)
- [Deep Video Prior for Video Consistency and Propagation](https://mlsanity.com/paper/2201.11632/) (2022)
- [PalGAN: Image Colorization with Palette Generative Adversarial Networks](https://mlsanity.com/paper/2210.11204/) (2022)
- [BiSTNet: Semantic Image Prior Guided Bidirectional Temporal Feature Fusion for Deep Exemplar-based Video Colorization](https://mlsanity.com/paper/2212.02268/) (2022)
- [VCGAN: Video Colorization with Hybrid Generative Adversarial Network](https://mlsanity.com/paper/2104.12357/) (2021)
- [Video Colorization with Pre-trained Text-to-Image Diffusion Models](https://mlsanity.com/paper/2306.01732/) (2023)
- [Follow-Your-Color: Multi-Instance Sketch Colorization](https://mlsanity.com/paper/2503.16948/) (2025)
- [Color2Embed: Fast Exemplar-Based Image Colorization using Color Embeddings](https://mlsanity.com/paper/2106.08017/) (2021)
- [Tracktention: Leveraging Point Tracking to Attend Videos Faster and Better](https://mlsanity.com/paper/2503.19904/) (2025)
- [SCSNet: An Efficient Paradigm for Learning Simultaneously Image Colorization and Super-Resolution](https://mlsanity.com/paper/2201.04364/) (2022)
- [ColorMNet: A Memory-based Deep Spatial-Temporal Feature Propagation Network for Video Colorization](https://mlsanity.com/paper/2404.06251/) (2024)
- [Colorizing Monochromatic Radiance Fields](https://mlsanity.com/paper/2402.12184/) (2024)
- [Automatic Controllable Colorization via Imagination](https://mlsanity.com/paper/2404.05661/) (2024)
- [ChromaDistill: Colorizing Monochrome Radiance Fields with Knowledge Distillation](https://mlsanity.com/paper/2309.07668/) (2023)
- [Exemplar-based Video Colorization with Long-term Spatiotemporal Dependency](https://mlsanity.com/paper/2303.15081/) (2023)
- [Learning Data-Driven Vector-Quantized Degradation Model for Animation Video Super-Resolution](https://mlsanity.com/paper/2303.09826/) (2023)
- [SVCNet: Scribble-based Video Colorization Network with Temporal Aggregation](https://mlsanity.com/paper/2303.11591/) (2023)
- [DiffColor: Toward High Fidelity Text-Guided Image Colorization with Diffusion Models](https://mlsanity.com/paper/2308.01655/) (2023)
- [Two Decades of Colorization and Decolorization for Images and Videos](https://mlsanity.com/paper/2204.13322/) (2022)
- [BBQ-to-Image: Numeric Bounding Box and Qolor Control in Large-Scale Text-to-Image Models](https://mlsanity.com/paper/2602.20672/) (2026)
- [TUCaN: Progressively Teaching Colourisation to Capsules](https://mlsanity.com/paper/2106.15176/) (2021)
- [Temporal Consistent Automatic Video Colorization via Semantic Correspondence](https://mlsanity.com/paper/2305.07904/) (2023)
- [L-C4: Language-Based Video Colorization for Creative and Consistent Color](https://mlsanity.com/paper/2410.04972/) (2024)
- [Consistent Video Colorization via Palette Guidance](https://mlsanity.com/paper/2501.19331/) (2025)
- [Collaboration among Image and Object Level Features for Image Colourisation](https://mlsanity.com/paper/2101.07576/) (2021)

## Abstract (arXiv)

We present a fully automatic approach to video colorization with self-regularization and diversity. Our model contains a colorization network for video frame colorization and a refinement network for spatiotemporal color refinement. Without any labeled data, both networks can be trained with self-regularized losses defined in bilateral and temporal space. The bilateral loss enforces color consistency between neighboring pixels in a bilateral space and the temporal loss imposes constraints between corresponding pixels in two nearby frames. While video colorization is a multi-modal problem, our method uses a perceptual loss with diversity to differentiate various modes in the solution space. Perceptual experiments demonstrate that our approach outperforms state-of-the-art approaches on fully automatic video colorization. The results are shown in the supplementary video at https://youtu.be/Y15uv2jnK-4