# Robust pedestrian detection in thermal imagery using synthesized images

arXiv: https://arxiv.org/abs/2102.02005 · HTML: https://mlsanity.com/paper/2102.02005/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 64 references that are in the map.
- First posted on arXiv 3 Feb 2021.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Autoencoding beyond pixels using a learned similarity metric](https://mlsanity.com/paper/1512.09300/) (2015)
- [Toward Multimodal Image-to-Image Translation](https://mlsanity.com/paper/1711.11586/) (2017)
- [Generating Images with Perceptual Similarity Metrics based on Deep Networks](https://mlsanity.com/paper/1602.02644/) (2016)
- [GAN Augmentation: Augmenting Training Data using Generative Adversarial Networks](https://mlsanity.com/paper/1810.10863/) (2018)
- [Multispectral Deep Neural Networks for Pedestrian Detection](https://mlsanity.com/paper/1611.02644/) (2016)
- [Illumination-aware Faster R-CNN for Robust Multispectral Pedestrian Detection](https://mlsanity.com/paper/1803.05347/) (2018)
- [Generative Multi-Adversarial Networks](https://mlsanity.com/paper/1611.01673/) (2016)
- [Fusion of Multispectral Data Through Illumination-aware Deep Neural Networks for Pedestrian Detection](https://mlsanity.com/paper/1802.09972/) (2018)
- [Multispectral Pedestrian Detection via Simultaneous Detection and Segmentation](https://mlsanity.com/paper/1808.04818/) (2018)
- [Deep Generative Adversarial Compression Artifact Removal](https://mlsanity.com/paper/1704.02518/) (2017)
- [Borrow from Anywhere: Pseudo Multi-modal Object Detection in Thermal Imagery](https://mlsanity.com/paper/1905.08789/) (2019)
- [Learning Cross-Modal Deep Representations for Robust Pedestrian Detection](https://mlsanity.com/paper/1704.02431/) (2017)
- [Synthetic data generation for end-to-end thermal infrared tracking](https://mlsanity.com/paper/1806.01013/) (2018)
- [Synthesis of High-Quality Visible Faces from Polarimetric Thermal Faces using Generative Adversarial Networks](https://mlsanity.com/paper/1812.05155/) (2018)
- [Pedestrian Detection aided by Deep Learning Semantic Tasks](https://mlsanity.com/paper/1412.0069/) (2014)
- [Pedestrian Detection in Thermal Images using Saliency Maps](https://mlsanity.com/paper/1904.06859/) (2019)

## Built on by (most cited first, 7 of 7)

- [Edge-guided Multi-domain RGB-to-TIR image Translation for Training Vision Tasks with Challenging Labels](https://mlsanity.com/paper/2301.12689/) (2023)
- [Pedestrian Detection in Low-Light Conditions: A Comprehensive Survey](https://mlsanity.com/paper/2401.07801/) (2024)
- [Mixture of Scale Experts for Alignment-free RGBT Video Object Detection and A Unified Benchmark](https://mlsanity.com/paper/2410.12143/) (2024)
- [Diffusion Dataset Generation: Towards Closing the Sim2Real Gap for Pedestrian Detection](https://mlsanity.com/paper/2305.09401/) (2023)
- [Partially fake it till you make it: mixing real and fake thermal images for improved object detection](https://mlsanity.com/paper/2106.13603/) (2021)
- [Data Generation Scheme for Thermal Modality with Edge-Guided Adversarial Conditional Diffusion Model](https://mlsanity.com/paper/2408.03748/) (2024)
- [Meta-Learning for Color-to-Infrared Cross-Modal Style Transfer](https://mlsanity.com/paper/2212.12824/) (2022)

## Abstract (arXiv)

In this paper we propose a method for improving pedestrian detection in the thermal domain using two stages: first, a generative data augmentation approach is used, then a domain adaptation method using generated data adapts an RGB pedestrian detector. Our model, based on the Least-Squares Generative Adversarial Network, is trained to synthesize realistic thermal versions of input RGB images which are then used to augment the limited amount of labeled thermal pedestrian images available for training. We apply our generative data augmentation strategy in order to adapt a pretrained YOLOv3 pedestrian detector to detection in the thermal-only domain. Experimental results demonstrate the effectiveness of our approach: using less than 50% of available real thermal training data, and relying on synthesized data generated by our model in the domain adaptation phase, our detector achieves state-of-the-art results on the KAIST Multispectral Pedestrian Detection Benchmark; even if more real thermal data is available adding GAN generated images to the training data results in improved performance, thus showing that these images act as an effective form of data augmentation. To the best of our knowledge, our detector achieves the best single-modality detection results on KAIST with respect to the state-of-the-art.