# Thermal Infrared Image Colorization for Nighttime Driving Scenes with Top-Down Guided Attention

arXiv: https://arxiv.org/abs/2104.14374 · HTML: https://mlsanity.com/paper/2104.14374/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 66 references that are in the map.
- First posted on arXiv 29 Apr 2021.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [DRIT++: Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1905.01270/) (2019)
- [The relativistic discriminator: a key element missing from standard GAN](https://mlsanity.com/paper/1807.00734/) (2018)
- [Asymmetric Non-local Neural Networks for Semantic Segmentation](https://mlsanity.com/paper/1908.07678/) (2019)
- [U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation](https://mlsanity.com/paper/1907.10830/) (2019)
- [Effectively Unbiased FID and Inception Score and where to find them](https://mlsanity.com/paper/1911.07023/) (2019)
- [Pyramid Feature Attention Network for Saliency detection](https://mlsanity.com/paper/1903.00179/) (2019)
- [SPAN: Spatial Pyramid Attention Network forImage Manipulation Localization](https://mlsanity.com/paper/2009.00726/) (2020)
- [Dense Attention Fluid Network for Salient Object Detection in Optical Remote Sensing Images](https://mlsanity.com/paper/2011.13144/) (2020)
- [Reusing Discriminators for Encoding: Towards Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/2003.00273/) (2020)
- [Night-to-Day Image Translation for Retrieval-based Localization](https://mlsanity.com/paper/1809.09767/) (2018)
- [Unsupervised Scene Adaptation with Memory Regularization in vivo](https://mlsanity.com/paper/1912.11164/) (2019)
- [Spatial Attention Pyramid Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2003.12979/) (2020)
- [Capturing Omni-Range Context for Omnidirectional Segmentation](https://mlsanity.com/paper/2103.05687/) (2021)
- [ComboGAN: Unrestrained Scalability for Image Domain Translation](https://mlsanity.com/paper/1712.06909/) (2017)
- [Segmenting Objects in Day and Night:Edge-Conditioned CNN for Thermal Image Semantic Segmentation](https://mlsanity.com/paper/1907.10303/) (2019)
- [Thermal Infrared Colorization via Conditional Generative Adversarial Network](https://mlsanity.com/paper/1810.05399/) (2018)
- [Unpaired Thermal to Visible Spectrum Transfer using Adversarial Training](https://mlsanity.com/paper/1904.02242/) (2019)
- [Learning Crisp Boundaries Using Deep Refinement Network and Adaptive Weighting Loss](https://mlsanity.com/paper/2102.01301/) (2021)
- [Semantic Adversarial Network with Multi-scale Pyramid Attention for Video Classification](https://mlsanity.com/paper/1903.02155/) (2019)

## Built on by (most cited first, 15 of 15)

- [Seeing Motion at Nighttime with an Event Camera](https://mlsanity.com/paper/2404.11884/) (2024)
- [Edge-guided Multi-domain RGB-to-TIR image Translation for Training Vision Tasks with Challenging Labels](https://mlsanity.com/paper/2301.12689/) (2023)
- [DMFourLLIE: Dual-Stage and Multi-Branch Fourier Network for Low-Light Image Enhancement](https://mlsanity.com/paper/2412.00683/) (2024)
- [Pedestrian Detection in Low-Light Conditions: A Comprehensive Survey](https://mlsanity.com/paper/2401.07801/) (2024)
- [StawGAN: Structural-Aware Generative Adversarial Networks for Infrared Image Translation](https://mlsanity.com/paper/2305.10882/) (2023)
- [DistillMatch: Leveraging Knowledge Distillation from Vision Foundation Model for Multimodal Image Matching](https://mlsanity.com/paper/2509.16017/) (2025)
- [Nighttime Thermal Infrared Image Colorization with Feedback-based Object Appearance Learning](https://mlsanity.com/paper/2310.15688/) (2023)
- [ShadowSense: Unsupervised Domain Adaptation and Feature Fusion for Shadow-Agnostic Tree Crown Detection from RGB-Thermal Drone Imagery](https://mlsanity.com/paper/2310.16212/) (2023)
- [CM-Diff: A Single Generative Network for Bidirectional Cross-Modality Translation Diffusion Model Between Infrared and Visible Images](https://mlsanity.com/paper/2503.09514/) (2025)
- [Memory-Guided Collaborative Attention for Nighttime Thermal Infrared Image Colorization](https://mlsanity.com/paper/2208.02960/) (2022)
- [Line Art Colorization of Fakemon using Generative Adversarial Neural Networks](https://mlsanity.com/paper/2307.05760/) (2023)
- [Contrast-Guided Cross-Modal Distillation for Thermal Object Detection](https://mlsanity.com/paper/2511.01435/) (2025)
- [A Novel Deep Learning Method for Thermal to Annotated Thermal-Optical Fused Images](https://mlsanity.com/paper/2107.05942/) (2021)
- [MMC: Multi-Modal Colorization of Images using Textual Descriptions](https://mlsanity.com/paper/2304.11993/) (2023)
- [LoGoColor: Local-Global 3D Colorization for 360deg Scenes](https://mlsanity.com/paper/2512.09278/) (2025)

## Abstract (arXiv)

Benefitting from insensitivity to light and high penetration of foggy environments, infrared cameras are widely used for sensing in nighttime traffic scenes. However, the low contrast and lack of chromaticity of thermal infrared (TIR) images hinder the human interpretation and portability of high-level computer vision algorithms. Colorization to translate a nighttime TIR image into a daytime color (NTIR2DC) image may be a promising way to facilitate nighttime scene perception. Despite recent impressive advances in image translation, semantic encoding entanglement and geometric distortion in the NTIR2DC task remain under-addressed. Hence, we propose a toP-down attEntion And gRadient aLignment based GAN, referred to as PearlGAN. A top-down guided attention module and an elaborate attentional loss are first designed to reduce the semantic encoding ambiguity during translation. Then, a structured gradient alignment loss is introduced to encourage edge consistency between the translated and input images. In addition, pixel-level annotation is carried out on a subset of FLIR and KAIST datasets to evaluate the semantic preservation performance of multiple translation methods. Furthermore, a new metric is devised to evaluate the geometric consistency in the translation process. Extensive experiments demonstrate the superiority of the proposed PearlGAN over other image translation methods for the NTIR2DC task. The source code and labeled segmentation masks will be available at urlhttps://github.com/FuyaLuo/PearlGAN/.