# Synthesis of High-Quality Visible Faces from Polarimetric Thermal Faces using Generative Adversarial Networks

arXiv: https://arxiv.org/abs/1812.05155 · HTML: https://mlsanity.com/paper/1812.05155/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Dec 2018.

## Built on by (most cited first, 24 of 24)

- [A Large-Scale, Time-Synchronized Visible and Thermal Face Dataset](https://mlsanity.com/paper/2101.02637/) (2021)
- [RoI Tanh-polar Transformer Network for Face Parsing in the Wild](https://mlsanity.com/paper/2102.02717/) (2021)
- [T2V-DDPM: Thermal to Visible Face Translation using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2209.08814/) (2022)
- [AGRNet: Adaptive Graph Representation Learning and Reasoning for Face Parsing](https://mlsanity.com/paper/2101.07034/) (2021)
- [A Study of the Human Perception of Synthetic Faces](https://mlsanity.com/paper/2111.04230/) (2021)
- [I2V-GAN: Unpaired Infrared-to-Visible Video Translation](https://mlsanity.com/paper/2108.00913/) (2021)
- [Simultaneous Face Hallucination and Translation for Thermal to Visible Face Verification using Axial-GAN](https://mlsanity.com/paper/2104.06534/) (2021)
- [Parameter Efficient Local Implicit Image Function Network for Face Segmentation](https://mlsanity.com/paper/2303.15122/) (2023)
- [Robust pedestrian detection in thermal imagery using synthesized images](https://mlsanity.com/paper/2102.02005/) (2021)
- [A Synthesis-Based Approach for Thermal-to-Visible Face Verification](https://mlsanity.com/paper/2108.09558/) (2021)
- [T-FAKE: Synthesizing Thermal Images for Facial Landmarking](https://mlsanity.com/paper/2408.15127/) (2024)
- [Beyond the Visible: A Survey on Cross-spectral Face Recognition](https://mlsanity.com/paper/2201.04435/) (2022)
- [Heterogeneous Face Frontalization via Domain Agnostic Learning](https://mlsanity.com/paper/2107.08311/) (2021)
- [Heterogeneous Face Recognition via Face Synthesis with Identity-Attribute Disentanglement](https://mlsanity.com/paper/2206.04854/) (2022)
- [When Visible-to-Thermal Facial GAN Beats Conditional Diffusion](https://mlsanity.com/paper/2302.09395/) (2023)
- [Physically-Based Face Rendering for NIR-VIS Face Recognition](https://mlsanity.com/paper/2211.06408/) (2022)
- [Multimodal Face Synthesis from Visual Attributes](https://mlsanity.com/paper/2104.04362/) (2021)
- [Joint Feature Distribution Alignment Learning for NIR-VIS and VIS-VIS Face Recognition](https://mlsanity.com/paper/2204.11434/) (2022)
- [Learning Domain and Pose Invariance for Thermal-to-Visible Face Recognition](https://mlsanity.com/paper/2211.09350/) (2022)
- [Generating Thermal Human Faces for Physiological Assessment Using Thermal Sensor Auxiliary Labels](https://mlsanity.com/paper/2106.08091/) (2021)
- [Thermal to Visible Image Synthesis under Atmospheric Turbulence](https://mlsanity.com/paper/2204.03057/) (2022)
- [Rethinking the Domain Gap in Near-infrared Face Recognition](https://mlsanity.com/paper/2312.00627/) (2023)
- [Escaping Data Scarcity for High-Resolution Heterogeneous Face Hallucination](https://mlsanity.com/paper/2203.16669/) (2022)
- [A Novel Fully Annotated Thermal Infrared Face Dataset: Recorded in Various Environment Conditions and Distances From The Camera](https://mlsanity.com/paper/2205.02093/) (2022)

## Abstract (arXiv)

The large domain discrepancy between faces captured in polarimetric (or conventional) thermal and visible domain makes cross-domain face verification a highly challenging problem for human examiners as well as computer vision algorithms. Previous approaches utilize either a two-step procedure (visible feature estimation and visible image reconstruction) or an input-level fusion technique, where different Stokes images are concatenated and used as a multi-channel input to synthesize the visible image given the corresponding polarimetric signatures. Although these methods have yielded improvements, we argue that input-level fusion alone may not be sufficient to realize the full potential of the available Stokes images. We propose a Generative Adversarial Networks (GAN) based multi-stream feature-level fusion technique to synthesize high-quality visible images from prolarimetric thermal images. The proposed network consists of a generator sub-network, constructed using an encoder-decoder network based on dense residual blocks, and a multi-scale discriminator sub-network. The generator network is trained by optimizing an adversarial loss in addition to a perceptual loss and an identity preserving loss to enable photo realistic generation of visible images while preserving discriminative characteristics. An extended dataset consisting of polarimetric thermal facial signatures of 111 subjects is also introduced. Multiple experiments evaluated on different experimental protocols demonstrate that the proposed method achieves state-of-the-art performance. Code will be made available at https://github.com/hezhangsprinter.