# VCGAN: Video Colorization with Hybrid Generative Adversarial Network

arXiv: https://arxiv.org/abs/2104.12357 · HTML: https://mlsanity.com/paper/2104.12357/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 54 references that are in the map.
- First posted on arXiv 26 Apr 2021.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Toward Multimodal Image-to-Image Translation](https://mlsanity.com/paper/1711.11586/) (2017)
- [Learning Blind Video Temporal Consistency](https://mlsanity.com/paper/1808.00449/) (2018)
- [Learning Representations for Automatic Colorization](https://mlsanity.com/paper/1603.06668/) (2016)
- [Real-Time User-Guided Image Colorization with Learned Deep Priors](https://mlsanity.com/paper/1705.02999/) (2017)
- [Deep Video Inpainting](https://mlsanity.com/paper/1905.01639/) (2019)
- [Instance-aware Image Colorization](https://mlsanity.com/paper/2005.10825/) (2020)
- [Deep Exemplar-based Video Colorization](https://mlsanity.com/paper/1906.09909/) (2019)
- [Deep Colorization](https://mlsanity.com/paper/1605.00075/) (2016)
- [ChromaGAN: Adversarial Picture Colorization with Semantic Class Distribution](https://mlsanity.com/paper/1907.09837/) (2019)
- [Artistic style transfer for videos](https://mlsanity.com/paper/1604.08610/) (2016)
- [Video Propagation Networks](https://mlsanity.com/paper/1612.05478/) (2016)
- [Learning Diverse Image Colorization](https://mlsanity.com/paper/1612.01958/) (2016)
- [Fully Automatic Video Colorization with Self-Regularization and Diversity](https://mlsanity.com/paper/1908.01311/) (2019)
- [PixColor: Pixel Recursive Colorization](https://mlsanity.com/paper/1705.07208/) (2017)
- [Unsupervised Diverse Colorization via Generative Adversarial Networks](https://mlsanity.com/paper/1702.06674/) (2017)
- [Pixelated Semantic Colorization](https://mlsanity.com/paper/1901.10889/) (2019)
- [SCGAN: Saliency Map-guided Colorization with Generative Adversarial Network](https://mlsanity.com/paper/2011.11377/) (2020)

## Built on by (most cited first, 12 of 12)

- [Video Colorization with Pre-trained Text-to-Image Diffusion Models](https://mlsanity.com/paper/2306.01732/) (2023)
- [ColorMNet: A Memory-based Deep Spatial-Temporal Feature Propagation Network for Video Colorization](https://mlsanity.com/paper/2404.06251/) (2024)
- [Exemplar-based Video Colorization with Long-term Spatiotemporal Dependency](https://mlsanity.com/paper/2303.15081/) (2023)
- [SVCNet: Scribble-based Video Colorization Network with Temporal Aggregation](https://mlsanity.com/paper/2303.11591/) (2023)
- [Temporal Consistent Automatic Video Colorization via Semantic Correspondence](https://mlsanity.com/paper/2305.07904/) (2023)
- [L-C4: Language-Based Video Colorization for Creative and Consistent Color](https://mlsanity.com/paper/2410.04972/) (2024)
- [Consistent Video Colorization via Palette Guidance](https://mlsanity.com/paper/2501.19331/) (2025)
- [Histogram-guided Video Colorization Structure with Spatial-Temporal Connection](https://mlsanity.com/paper/2308.04899/) (2023)
- [LatentColorization: Latent Diffusion-Based Speaker Video Colorization](https://mlsanity.com/paper/2405.05707/) (2024)
- [Adapting MIMO video restoration networks to low latency constraints](https://mlsanity.com/paper/2408.12439/) (2024)
- [Instant Video Models: Universal Adapters for Stabilizing Image-Based Networks](https://mlsanity.com/paper/2512.03014/) (2025)
- [VibeFlow: Versatile Video Chroma-Lux Editing through Self-Supervised Learning](https://mlsanity.com/paper/2604.13425/) (2026)

## Abstract (arXiv)

We propose a hybrid recurrent Video Colorization with Hybrid Generative Adversarial Network (VCGAN), an improved approach to video colorization using end-to-end learning. The VCGAN addresses two prevalent issues in the video colorization domain: Temporal consistency and unification of colorization network and refinement network into a single architecture. To enhance colorization quality and spatiotemporal consistency, the mainstream of generator in VCGAN is assisted by two additional networks, i.e., global feature extractor and placeholder feature extractor, respectively. The global feature extractor encodes the global semantics of grayscale input to enhance colorization quality, whereas the placeholder feature extractor acts as a feedback connection to encode the semantics of the previous colorized frame in order to maintain spatiotemporal consistency. If changing the input for placeholder feature extractor as grayscale input, the hybrid VCGAN also has the potential to perform image colorization. To improve the consistency of far frames, we propose a dense long-term loss that smooths the temporal disparity of every two remote frames. Trained with colorization and temporal losses jointly, VCGAN strikes a good balance between color vividness and video continuity. Experimental results demonstrate that VCGAN produces higher-quality and temporally more consistent colorful videos than existing approaches.