mlsanityData to 7 Oct 2026

Colorization Transformer

arXiv 2102.04432 ↗First posted 8 Feb 2021cs.CV cs.AI cs.LGcited by 89

Not on the map. Why?

Abstract (arXiv)

We present the Colorization Transformer, a novel approach for diverse high fidelity image colorization based on self-attention. Given a grayscale image, the colorization proceeds in three steps. We first use a conditional autoregressive transformer to produce a low resolution coarse coloring of the grayscale image. Our architecture adopts conditional transformer layers to effectively condition grayscale input. Two subsequent fully parallel networks upsample the coarse colored low resolution image into a finely colored high resolution image. Sampling from the Colorization Transformer produces diverse colorings whose fidelity outperforms the previous state-of-the-art on colorising ImageNet based on FID results and based on a human evaluation in a Mechanical Turk test. Remarkably, in more than 60% of cases human evaluators prefer the highest rated among three generated colorings over the ground truth. The code and pre-trained checkpoints for Colorization Transformer are publicly available at https://github.com/google-research/google-research/tree/master/coltran

Key facts. Cited by 89 papers in the map (data through 7 Oct 2026). Builds on 35 of its 58 references that are in the map. First posted on arXiv 8 Feb 2021.

Builds on

35 of 58 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2016Layer Normalizationcited by 5,579
  4. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  5. 2013Auto-Encoding Variational Bayescited by 3,980
  6. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  7. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  8. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  9. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  10. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  11. 2016Colorful Image Colorizationcited by 1,113
  12. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
23 more
  1. 2016Pixel Recurrent Neural Networkscited by 814
  2. 2018Self-Attention Generative Adversarial Networkscited by 809
  3. 2018Image Transformercited by 770
  4. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  5. 2019Axial Attention in Multidimensional Transformerscited by 350
  6. 2016A Learned Representation For Artistic Stylecited by 340
  7. 2020Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentationcited by 326
  8. 2019Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Designcited by 258
  9. 2016Learning Representations for Automatic Colorizationcited by 251
  10. 2019Scaling Autoregressive Video Modelscited by 189
  11. 2017Modulating early visual processing by languagecited by 167
  12. 2018Tracking Emerges by Colorizing Videoscited by 166
  13. 2019Guided Image Generation with Conditional Invertible Neural Networkscited by 147
  14. 2017Real-Time User-Guided Image Colorization with Learned Deep Priorscited by 123
  15. 2019Deep Exemplar-based Video Colorizationcited by 75
  16. 2018Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscalingcited by 74
  17. 2016Deep Colorizationcited by 70
  18. 2016Learning Diverse Image Colorizationcited by 43
  19. 2017PixColor: Pixel Recursive Colorizationcited by 32
  20. 2018User-Guided Deep Anime Line Art Colorization with Conditional Adversarial Networkscited by 32
  21. 2017Unsupervised Diverse Colorization via Generative Adversarial Networkscited by 30
  22. 2018Structural Consistency and Controllability for Diverse Colorizationcited by 15
  23. 2017Probabilistic Image Colorizationcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 24; 2023: 23; 2024: 13; 2025: 9; 2026: 1.

89 papers.

Sort:

By area:

  1. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  2. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,618
  3. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  4. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
  5. 2022Transformers in Medical Imaging: A Surveycited by 245
  6. 2021TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Upcited by 162
  7. 2021Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimationcited by 122
  8. 2022Accurate Image Restoration with Attention Retractable Transformercited by 85
  9. 2022Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encodingcited by 83
  10. 2022You Only Need 90K Parameters to Adapt Light: A Light Weight Transformer for Image Enhancement and Exposure Correctioncited by 81
  11. 2022UViM: A Unified Modeling Approach for Vision with Learned Guiding Codescited by 77
  12. 2023Towards Accurate Post-Training Quantization for Vision Transformercited by 70
38 more
  1. 2021Gaze Estimation using Transformercited by 68
  2. 2021Keyword Transformer: A Self-Attention Model for Keyword Spottingcited by 59
  3. 2021Towards Vivid and Diverse Image Colorization with Generative Color Priorcited by 55
  4. 2021Perceptual Image Quality Assessment with Transformerscited by 54
  5. 2021ISTR: End-to-End Instance Segmentation with Transformerscited by 52
  6. 2023WaveDM: Wavelet-Based Diffusion Models for Image Restorationcited by 46
  7. 2022Toward Real-world Single Image Deraining: A New Benchmark and Beyondcited by 40
  8. 2022DDColor: Towards Photo-Realistic Image Colorization via Dual Decoderscited by 36
  9. 2022BigColor: Colorization using a Generative Color Prior for Natural Imagescited by 34
  10. 2024Control Color: Multimodal Diffusion-based Interactive Image Colorizationcited by 33
  11. 2021Eformer: Edge Enhancement based Transformer for Medical Image Denoisingcited by 30
  12. 2023Coarse-to-Fine Amodal Segmentation with Shape Priorcited by 29
  13. 2023Object Detection with Transformers: A Reviewcited by 22
  14. 2021Temporal Action Proposal Generation with Transformerscited by 17
  15. 2021Pretrained Language Models are Symbolic Mathematics Solvers too!cited by 17
  16. 2022PalGAN: Image Colorization with Palette Generative Adversarial Networkscited by 15
  17. 2023Improved Diffusion-based Image Colorization via Piggybacked Modelscited by 15
  18. 2021The Image Local Autoregressive Transformercited by 14
  19. 2021Semantic-Sparse Colorization Network for Deep Exemplar-based Colorizationcited by 12
  20. 2023Video Colorization with Pre-trained Text-to-Image Diffusion Modelscited by 11
  21. 2024Tackling the Abstraction and Reasoning Corpus with Vision Transformers: the Importance of 2D Representation, Positions, and Objectscited by 9
  22. 2021Attention mechanisms and deep learning for machine vision: A survey of the state of the artcited by 8
  23. 2022SCSNet: An Efficient Paradigm for Learning Simultaneously Image Colorization and Super-Resolutioncited by 8
  24. 2022Generative Colorization of Structured Mobile Web Pagescited by 8
  25. 2022UniColor: A Unified Framework for Multi-Modal Colorization with Transformercited by 7
  26. 2024Automatic Controllable Colorization via Imaginationcited by 7
  27. 2023Cooperative Colorization: Exploring Latent Cross-Domain Priors for NIR Image Spectrum Translationcited by 6
  28. 2024MultiColor: Image Colorization by Learning from Multiple Color Spacescited by 6
  29. 2021I2C2W: Image-to-Character-to-Word Transformers for Accurate Scene Text Recognitioncited by 5
  30. 2022Attention-Aware Anime Line Drawing Colorizationcited by 5
  31. 2023L-CAD: Language-based Colorization with Any-level Descriptions using Diffusion Priorscited by 5
  32. 2023Pre-Training LiDAR-Based 3D Object Detectors Through Colorizationcited by 5
  33. 2023Exemplar-based Video Colorization with Long-term Spatiotemporal Dependencycited by 4
  34. 2023StawGAN: Structural-Aware Generative Adversarial Networks for Infrared Image Translationcited by 4
  35. 2022StyleFlow For Content-Fixed Image to Image Translationcited by 3
  36. 2023Simulating analogue film damage to analyse and improve artefact restoration on high-resolution scanscited by 3
  37. 2023SVCNet: Scribble-based Video Colorization Network with Temporal Aggregationcited by 3
  38. 2023SPColor: Semantic Prior Guided Exemplar-based Image Colorizationcited by 3

50 most cited of 89 All 89 (JSON)