# Wavelet Transform-assisted Adaptive Generative Modeling for Colorization

arXiv: https://arxiv.org/abs/2107.04261 · HTML: https://mlsanity.com/paper/2107.04261/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 77 references that are in the map.
- First posted on arXiv 9 Jul 2021.

## Builds on (most cited first, 22 of 22)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation](https://mlsanity.com/paper/1611.06612/) (2016)
- [Image Processing Using Multi-Code GAN Prior](https://mlsanity.com/paper/1912.07116/) (2019)
- [The Fast Bilateral Solver](https://mlsanity.com/paper/1511.03296/) (2015)
- [ChromaGAN: Adversarial Picture Colorization with Semantic Class Distribution](https://mlsanity.com/paper/1907.09837/) (2019)
- [Learning Diverse Image Colorization](https://mlsanity.com/paper/1612.01958/) (2016)
- [Source Separation with Deep Generative Priors](https://mlsanity.com/paper/2002.07942/) (2020)
- [Coloring With Limited Data: Few-Shot Colorization via Memory-Augmented Networks](https://mlsanity.com/paper/1906.11888/) (2019)
- [Unsupervised Diverse Colorization via Generative Adversarial Networks](https://mlsanity.com/paper/1702.06674/) (2017)
- [Pixelated Semantic Colorization](https://mlsanity.com/paper/1901.10889/) (2019)
- [Image Colorization: A Survey and Dataset](https://mlsanity.com/paper/2008.10774/) (2020)
- [Homotopic Gradients of Generative Density Priors for MR Image Reconstruction](https://mlsanity.com/paper/2008.06284/) (2020)
- [Fast Mixing of Multi-Scale Langevin Dynamics under the Manifold Hypothesis](https://mlsanity.com/paper/2006.11166/) (2020)
- [Efficient and principled score estimation with Nystr"om kernel exponential families](https://mlsanity.com/paper/1705.08360/) (2017)
- [Learning High-Dimensional Distributions with Latent Neural Fokker-Planck Kernels](https://mlsanity.com/paper/2105.04538/) (2021)
- [Fast Bilateral Filtering of Vector-Valued Images](https://mlsanity.com/paper/1605.02164/) (2016)

## Built on by (most cited first, 8 of 8)

- [Wavelet Diffusion Models are fast and scalable Image Generators](https://mlsanity.com/paper/2211.16152/) (2022)
- [Boosting Diffusion Models with Moving Average Sampling in Frequency Domain](https://mlsanity.com/paper/2403.17870/) (2024)
- [Stage-by-stage Wavelet Optimization Refinement Diffusion Model for Sparse-View CT Reconstruction](https://mlsanity.com/paper/2308.15942/) (2023)
- [Fr'echet Wavelet Distance: A Domain-Agnostic Metric for Image Generation](https://mlsanity.com/paper/2312.15289/) (2023)
- [Frequency Regulation for Exposure Bias Mitigation in Diffusion Models](https://mlsanity.com/paper/2507.10072/) (2025)
- [Exemplar-based Video Colorization with Long-term Spatiotemporal Dependency](https://mlsanity.com/paper/2303.15081/) (2023)
- [SPColor: Semantic Prior Guided Exemplar-based Image Colorization](https://mlsanity.com/paper/2304.06255/) (2023)
- [Generative Modeling in Structural-Hankel Domain for Color Image Inpainting](https://mlsanity.com/paper/2211.13857/) (2022)

## Abstract (arXiv)

Unsupervised deep learning has recently demonstrated the promise of producing high-quality samples. While it has tremendous potential to promote the image colorization task, the performance is limited owing to the high-dimension of data manifold and model capability. This study presents a novel scheme that exploits the score-based generative model in wavelet domain to address the issues. By taking advantage of the multi-scale and multi-channel representation via wavelet transform, the proposed model learns the richer priors from stacked coarse and detailed wavelet coefficient components jointly and effectively. This strategy also reduces the dimension of the original manifold and alleviates the curse of dimensionality, which is beneficial for estimation and sampling. Moreover, dual consistency terms in the wavelet domain, namely data-consistency and structure-consistency are devised to leverage colorization task better. Specifically, in the training phase, a set of multi-channel tensors consisting of wavelet coefficients is used as the input to train the network with denoising score matching. In the inference phase, samples are iteratively generated via annealed Langevin dynamics with data and structure consistencies. Experiments demonstrated remarkable improvements of the proposed method on both generation and colorization quality, particularly in colorization robustness and diversity.