# A Synthesis-Based Approach for Thermal-to-Visible Face Verification

arXiv: https://arxiv.org/abs/2108.09558 · HTML: https://mlsanity.com/paper/2108.09558/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 38 references that are in the map.
- First posted on arXiv 21 Aug 2021.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Bag of Tricks and A Strong Baseline for Deep Person Re-identification](https://mlsanity.com/paper/1903.07071/) (2019)
- [Toward Multimodal Image-to-Image Translation](https://mlsanity.com/paper/1711.11586/) (2017)
- [Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis](https://mlsanity.com/paper/1704.04086/) (2017)
- [An All-In-One Convolutional Neural Network for Face Analysis](https://mlsanity.com/paper/1611.00851/) (2016)
- [Deep Alignment Network: A convolutional neural network for robust face alignment](https://mlsanity.com/paper/1706.01789/) (2017)
- [The Devil is in the Details: Self-Supervised Attention for Vehicle Re-Identification](https://mlsanity.com/paper/2004.06271/) (2020)
- [A Dual-Path Model With Adaptive Attention For Vehicle Re-Identification](https://mlsanity.com/paper/1905.03397/) (2019)
- [Synthesis of High-Quality Visible Faces from Polarimetric Thermal Faces using Generative Adversarial Networks](https://mlsanity.com/paper/1812.05155/) (2018)
- [A Large-Scale, Time-Synchronized Visible and Thermal Face Dataset](https://mlsanity.com/paper/2101.02637/) (2021)
- [Polarimetric Thermal to Visible Face Verification via Self-Attention Guided Synthesis](https://mlsanity.com/paper/1904.07344/) (2019)
- [Cross-Domain Identification for Thermal-to-Visible Face Recognition](https://mlsanity.com/paper/2008.08473/) (2020)
- [The Do's and Don'ts for CNN-based Face Verification](https://mlsanity.com/paper/1705.07426/) (2017)
- [Dual-Attention GAN for Large-Pose Face Frontalization](https://mlsanity.com/paper/2002.07227/) (2020)
- [Multi-Scale Thermal to Visible Face Verification via Attribute Guided Synthesis](https://mlsanity.com/paper/2004.09502/) (2020)
- [M2FPA: A Multi-Yaw Multi-Pitch High-Quality Database and Benchmark for Facial Pose Analysis](https://mlsanity.com/paper/1904.00168/) (2019)
- [Heterogeneous Face Frontalization via Domain Agnostic Learning](https://mlsanity.com/paper/2107.08311/) (2021)

## Built on by (most cited first, 7 of 7)

- [T2V-DDPM: Thermal to Visible Face Translation using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2209.08814/) (2022)
- [Beyond the Visible: A Survey on Cross-spectral Face Recognition](https://mlsanity.com/paper/2201.04435/) (2022)
- [A Brief Survey on Person Recognition at a Distance](https://mlsanity.com/paper/2212.08969/) (2022)
- [Template-based Multi-Domain Face Recognition](https://mlsanity.com/paper/2409.09832/) (2024)
- [Learning Domain and Pose Invariance for Thermal-to-Visible Face Recognition](https://mlsanity.com/paper/2211.09350/) (2022)
- [Multi-Domain Biometric Recognition using Body Embeddings](https://mlsanity.com/paper/2503.10931/) (2025)
- [Escaping Data Scarcity for High-Resolution Heterogeneous Face Hallucination](https://mlsanity.com/paper/2203.16669/) (2022)

## Abstract (arXiv)

In recent years, visible-spectrum face verification systems have been shown to match the performance of experienced forensic examiners. However, such systems are ineffective in low-light and nighttime conditions. Thermal face imagery, which captures body heat emissions, effectively augments the visible spectrum, capturing discriminative facial features in scenes with limited illumination. Due to the increased cost and difficulty of obtaining diverse, paired thermal and visible spectrum datasets, not many algorithms and large-scale benchmarks for low-light recognition are available. This paper presents an algorithm that achieves state-of-the-art performance on both the ARL-VTF and TUFTS multi-spectral face datasets. Importantly, we study the impact of face alignment, pixel-level correspondence, and identity classification with label smoothing for multi-spectral face synthesis and verification. We show that our proposed method is widely applicable, robust, and highly effective. In addition, we show that the proposed method significantly outperforms face frontalization methods on profile-to-frontal verification. Finally, we present MILAB-VTF(B), a challenging multi-spectral face dataset that is composed of paired thermal and visible videos. To the best of our knowledge, with face data from 400 subjects, this dataset represents the most extensive collection of indoor and long-range outdoor thermal-visible face imagery. Lastly, we show that our end-to-end thermal-to-visible face verification system provides strong performance on the MILAB-VTF(B) dataset.