# DDColor: Towards Photo-Realistic Image Colorization via Dual Decoders

arXiv: https://arxiv.org/abs/2212.11613 · HTML: https://mlsanity.com/paper/2212.11613/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 54 references that are in the map.
- First posted on arXiv 22 Dec 2022.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [TransTrack: Multiple Object Tracking with Transformer](https://mlsanity.com/paper/2012.15460/) (2020)
- [Learning Deconvolution Network for Semantic Segmentation](https://mlsanity.com/paper/1505.04366/) (2015)
- [Instances as Queries](https://mlsanity.com/paper/2105.01928/) (2021)
- [Guided Image Generation with Conditional Invertible Neural Networks](https://mlsanity.com/paper/1907.02392/) (2019)
- [Colorization Transformer](https://mlsanity.com/paper/2102.04432/) (2021)
- [Deep Exemplar-based Colorization](https://mlsanity.com/paper/1807.06587/) (2018)
- [Instance-aware Image Colorization](https://mlsanity.com/paper/2005.10825/) (2020)
- [Deep Colorization](https://mlsanity.com/paper/1605.00075/) (2016)
- [ChromaGAN: Adversarial Picture Colorization with Semantic Class Distribution](https://mlsanity.com/paper/1907.09837/) (2019)
- [Towards Vivid and Diverse Image Colorization with Generative Color Prior](https://mlsanity.com/paper/2108.08826/) (2021)
- [Learning Diverse Image Colorization](https://mlsanity.com/paper/1612.01958/) (2016)

## Built on by (most cited first, 30 of 36)

- [Control Color: Multimodal Diffusion-based Interactive Image Colorization](https://mlsanity.com/paper/2402.10855/) (2024)
- [Lift3D: Zero-Shot Lifting of Any 2D Vision Model to 3D](https://mlsanity.com/paper/2403.18922/) (2024)
- [Tracktention: Leveraging Point Tracking to Attend Videos Faster and Better](https://mlsanity.com/paper/2503.19904/) (2025)
- [ColorMNet: A Memory-based Deep Spatial-Temporal Feature Propagation Network for Video Colorization](https://mlsanity.com/paper/2404.06251/) (2024)
- [Automatic Controllable Colorization via Imagination](https://mlsanity.com/paper/2404.05661/) (2024)
- [MultiColor: Image Colorization by Learning from Multiple Color Spaces](https://mlsanity.com/paper/2408.04172/) (2024)
- [ChromaDistill: Colorizing Monochrome Radiance Fields with Knowledge Distillation](https://mlsanity.com/paper/2309.07668/) (2023)
- [SelfAge: Personalized Facial Age Transformation Using Self-reference Images](https://mlsanity.com/paper/2502.13987/) (2025)
- [Color3D: Controllable and Consistent 3D Colorization with Personalized Colorizer](https://mlsanity.com/paper/2510.10152/) (2025)
- [ColorFLUX: A Structure-Color Decoupling Framework for Old Photo Colorization](https://mlsanity.com/paper/2603.28162/) (2026)
- [CCC: Color Classified Colorization](https://mlsanity.com/paper/2403.01476/) (2024)
- [Transforming Color: A Novel Image Colorization Method](https://mlsanity.com/paper/2410.04799/) (2024)
- [Consistent Video Colorization via Palette Guidance](https://mlsanity.com/paper/2501.19331/) (2025)
- [Image Super-Resolution with Guarantees via Conformalized Generative Models](https://mlsanity.com/paper/2502.09664/) (2025)
- [Language-based Image Colorization: A Benchmark and Beyond](https://mlsanity.com/paper/2503.14974/) (2025)
- [ColorVein: Colorful Cancelable Vein Biometrics](https://mlsanity.com/paper/2504.14253/) (2025)
- [Uncolorable Examples: Preventing Unauthorized AI Colorization via Perception-Aware Chroma-Restrictive Perturbation](https://mlsanity.com/paper/2510.08979/) (2025)
- [PRevivor: Reviving Ancient Chinese Paintings using Prior-Guided Color Transformers](https://mlsanity.com/paper/2511.01274/) (2025)
- [RDFace: A Benchmark Dataset for Rare Disease Facial Image Analysis under Extreme Data Scarcity and Phenotype-Aware Synthetic Generation](https://mlsanity.com/paper/2604.03454/) (2026)
- [CCC++: Optimized Color Classified Colorization with Segment Anything Model (SAM) Empowered Object Selective Color Harmonization](https://mlsanity.com/paper/2403.11494/) (2024)
- [ControlCol: Controllability in Automatic Speaker Video Colorization](https://mlsanity.com/paper/2408.11711/) (2024)
- [Computer-aided Colorization State-of-the-science: A Survey](https://mlsanity.com/paper/2410.02288/) (2024)
- [Preserving Old Memories in Vivid Detail: Human-Interactive Photo Restoration Framework](https://mlsanity.com/paper/2410.09529/) (2024)
- [A GAN-Enhanced Deep Learning Framework for Rooftop Detection from Historical Aerial Imagery](https://mlsanity.com/paper/2503.23200/) (2025)
- [Instance-aware Image Colorization with Controllable Textual Descriptions and Segmentation Masks](https://mlsanity.com/paper/2505.08705/) (2025)
- [Leveraging the Powerful Attention of a Pre-trained Diffusion Model for Exemplar-based Image Colorization](https://mlsanity.com/paper/2505.15812/) (2025)
- [PhotonSplat: 3D Scene Reconstruction and Colorization from SPAD Sensors](https://mlsanity.com/paper/2506.21680/) (2025)
- [Color and Frequency Correction for Image Colorization](https://mlsanity.com/paper/2510.23399/) (2025)
- [LoGoColor: Local-Global 3D Colorization for 360deg Scenes](https://mlsanity.com/paper/2512.09278/) (2025)
- [Color When It Counts: Grayscale-Guided Online Triggering for Always-On Streaming Video Sensing](https://mlsanity.com/paper/2603.22466/) (2026)

## Abstract (arXiv)

Image colorization is a challenging problem due to multi-modal uncertainty and high ill-posedness. Directly training a deep neural network usually leads to incorrect semantic colors and low color richness. While transformer-based methods can deliver better results, they often rely on manually designed priors, suffer from poor generalization ability, and introduce color bleeding effects. To address these issues, we propose DDColor, an end-to-end method with dual decoders for image colorization. Our approach includes a pixel decoder and a query-based color decoder. The former restores the spatial resolution of the image, while the latter utilizes rich visual features to refine color queries, thus avoiding hand-crafted priors. Our two decoders work together to establish correlations between color and multi-scale semantic representations via cross-attention, significantly alleviating the color bleeding effect. Additionally, a simple yet effective colorfulness loss is introduced to enhance the color richness. Extensive experiments demonstrate that DDColor achieves superior performance to existing state-of-the-art works both quantitatively and qualitatively. The codes and models are publicly available at https://github.com/piddnad/DDColor.