# Edge-guided Multi-domain RGB-to-TIR image Translation for Training Vision Tasks with Challenging Labels

arXiv: https://arxiv.org/abs/2301.12689 · HTML: https://mlsanity.com/paper/2301.12689/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 44 references that are in the map.
- First posted on arXiv 30 Jan 2023.

## Builds on (most cited first, 24 of 24)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [DRIT++: Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1905.01270/) (2019)
- [VarifocalNet: An IoU-aware Dense Object Detector](https://mlsanity.com/paper/2008.13367/) (2020)
- [Learning to Estimate Hidden Motions with Global Motion Aggregation](https://mlsanity.com/paper/2104.02409/) (2021)
- [Driving in the Matrix: Can Virtual Worlds Replace Human-Generated Annotations for Real World Tasks?](https://mlsanity.com/paper/1610.01983/) (2016)
- [Playing for Benchmarks](https://mlsanity.com/paper/1709.07322/) (2017)
- [Dual Contrastive Learning for Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/2104.07689/) (2021)
- [Borrow from Anywhere: Pseudo Multi-modal Object Detection in Thermal Imagery](https://mlsanity.com/paper/1905.08789/) (2019)
- [Segmenting Objects in Day and Night:Edge-Conditioned CNN for Thermal Image Semantic Segmentation](https://mlsanity.com/paper/1907.10303/) (2019)
- [Synthetic data generation for end-to-end thermal infrared tracking](https://mlsanity.com/paper/1806.01013/) (2018)
- [Temporal Coherence for Active Learning in Videos](https://mlsanity.com/paper/1908.11757/) (2019)
- [Thermal Infrared Colorization via Conditional Generative Adversarial Network](https://mlsanity.com/paper/1810.05399/) (2018)
- [DeepTIO: A Deep Thermal-Inertial Odometry with Visual Hallucination](https://mlsanity.com/paper/1909.07231/) (2019)
- [Thermal Infrared Image Colorization for Nighttime Driving Scenes with Top-Down Guided Attention](https://mlsanity.com/paper/2104.14374/) (2021)
- [Optical Flow Estimation in the Deep Learning Age](https://mlsanity.com/paper/2004.02853/) (2020)
- [Graph-based Thermal-Inertial SLAM with Probabilistic Neural Networks](https://mlsanity.com/paper/2104.07196/) (2021)
- [Unpaired Thermal to Visible Spectrum Transfer using Adversarial Training](https://mlsanity.com/paper/1904.02242/) (2019)
- [Robust pedestrian detection in thermal imagery using synthesized images](https://mlsanity.com/paper/2102.02005/) (2021)
- [Online Photometric Calibration of Automatic Gain Thermal Infrared Cameras](https://mlsanity.com/paper/2012.14292/) (2020)

## Built on by (most cited first, 30 of 30)

- [LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment](https://mlsanity.com/paper/2310.01852/) (2023)
- [XoFTR: Cross-modal Feature Matching Transformer](https://mlsanity.com/paper/2404.09692/) (2024)
- [Complementary Random Masking for RGB-Thermal Semantic Segmentation](https://mlsanity.com/paper/2303.17386/) (2023)
- [From Specific-MLLMs to Omni-MLLMs: A Survey on MLLMs Aligned with Multi-modalities](https://mlsanity.com/paper/2412.11694/) (2024)
- [Caltech Aerial RGB-Thermal Dataset in the Wild](https://mlsanity.com/paper/2403.08997/) (2024)
- [TFDet: Target-Aware Fusion for RGB-T Pedestrian Detection](https://mlsanity.com/paper/2305.16580/) (2023)
- [PID: Physics-Informed Diffusion Model for Infrared Image Generation](https://mlsanity.com/paper/2407.09299/) (2024)
- [DiffV2IR: Visible-to-Infrared Diffusion Model via Vision-Language Understanding](https://mlsanity.com/paper/2503.19012/) (2025)
- [DMFourLLIE: Dual-Stage and Multi-Branch Fourier Network for Low-Light Image Enhancement](https://mlsanity.com/paper/2412.00683/) (2024)
- [ThermalGen: Style-Disentangled Flow-Based Generative Models for RGB-to-Thermal Image Translation](https://mlsanity.com/paper/2509.24878/) (2025)
- [IRGPT: Understanding Real-world Infrared Image with Bi-cross-modal Curriculum on Large-scale Benchmark](https://mlsanity.com/paper/2507.14449/) (2025)
- [F-ViTA: Foundation Model Guided Visible to Thermal Translation](https://mlsanity.com/paper/2504.02801/) (2025)
- [VIFNet: An End-to-end Visible-Infrared Fusion Network for Image Dehazing](https://mlsanity.com/paper/2404.07790/) (2024)
- [TherA: Thermal-Aware Visual-Language Prompting for Controllable RGB-to-Thermal Infrared Translation](https://mlsanity.com/paper/2602.19430/) (2026)
- [Cross-Modal Spherical Aggregation for Weakly Supervised Remote Sensing Shadow Removal](https://mlsanity.com/paper/2406.17469/) (2024)
- [TUNI: Unifying Pre-training and Fine-tuning with Modality-Aware Mutual Learning and Rectification for RGB-T Semantic Segmentation](https://mlsanity.com/paper/2509.10005/) (2025)
- [FusionRS: A Large-Scale RGB-Infrared-Style Remote Sensing Dataset for Cross-Modal Vision-Language Learning](https://mlsanity.com/paper/2606.17020/) (2026)
- [OSSA: Unsupervised One-Shot Style Adaptation](https://mlsanity.com/paper/2410.00900/) (2024)
- [CapHDR2IR: Caption-Driven Transfer from Visible Light to Infrared Domain](https://mlsanity.com/paper/2411.16327/) (2024)
- [UTA-Sign: Unsupervised Thermal Video Augmentation via Event-Assisted Traffic Signage Sketching](https://mlsanity.com/paper/2508.20594/) (2025)
- [Multi-Modal Camera-Based Detection of Vulnerable Road Users](https://mlsanity.com/paper/2509.06333/) (2025)
- [Enabling Plant Phenotyping in Weedy Environments using Multi-Modal Imagery via Synthetic and Generated Training Data](https://mlsanity.com/paper/2509.19208/) (2025)
- [Missing No More: Dictionary-Guided Cross-Modal Image Fusion under Missing Infrared](https://mlsanity.com/paper/2603.08018/) (2026)
- [UniTriGen: Unified Triplet Generation of Aligned Visible-Infrared-Label for Few-Shot RGB-T Semantic Segmentation](https://mlsanity.com/paper/2605.14626/) (2026)
- [Addressing Detail Bottlenecks in Latent Diffusion for RGB-to-SWIR Image Translation](https://mlsanity.com/paper/2606.19961/) (2026)
- [Enabling 24-hour Agricultural Robotics: Unsupervised Day-to-Night Cross-Modal Image Translation for Nighttime Visual Navigation](https://mlsanity.com/paper/2607.12065/) (2026)
- [G-MAD: A Game-Based Data Generation Framework for Multi-View RGB-T Aerial Object Detection](https://mlsanity.com/paper/2607.19942/) (2026)
- [SC-Diff: Semantically Calibrated Diffusion for Visible-to-Infrared Image Translation](https://mlsanity.com/paper/2608.08555/) (2026)
- [RGB-to-IR image translation for infrared vehicle detection in unseen UAV domains](https://mlsanity.com/paper/2609.02556/) (2026)
- [Text2Thermal: Physics-Aware Thermal Image Synthesis from Textual Priors](https://mlsanity.com/paper/2609.03585/) (2026)

## Abstract (arXiv)

The insufficient number of annotated thermal infrared (TIR) image datasets not only hinders TIR image-based deep learning networks to have comparable performances to that of RGB but it also limits the supervised learning of TIR image-based tasks with challenging labels. As a remedy, we propose a modified multidomain RGB to TIR image translation model focused on edge preservation to employ annotated RGB images with challenging labels. Our proposed method not only preserves key details in the original image but also leverages the optimal TIR style code to portray accurate TIR characteristics in the translated image, when applied on both synthetic and real world RGB images. Using our translation model, we have enabled the supervised learning of deep TIR image-based optical flow estimation and object detection that ameliorated in deep TIR optical flow estimation by reduction in end point error by 56.5% on average and the best object detection mAP of 23.9% respectively. Our code and supplementary materials are available at https://github.com/rpmsnu/sRGB-TIR.