# Generating Thermal Human Faces for Physiological Assessment Using Thermal Sensor Auxiliary Labels

arXiv: https://arxiv.org/abs/2106.08091 · HTML: https://mlsanity.com/paper/2106.08091/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 15 Jun 2021.

## Builds on (most cited first, 15 of 15)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Toward Multimodal Image-to-Image Translation](https://mlsanity.com/paper/1711.11586/) (2017)
- [Adversarial Uni- and Multi-modal Stream Networks for Multimodal Image Registration](https://mlsanity.com/paper/2007.02790/) (2020)
- [Synthesis of High-Quality Visible Faces from Polarimetric Thermal Faces using Generative Adversarial Networks](https://mlsanity.com/paper/1812.05155/) (2018)
- [Thermal Infrared Colorization via Conditional Generative Adversarial Network](https://mlsanity.com/paper/1810.05399/) (2018)
- [TV-GAN: Generative Adversarial Network Based Thermal to Visible Face Recognition](https://mlsanity.com/paper/1712.02514/) (2017)
- [PCSGAN: Perceptual Cyclic-Synthesized Generative Adversarial Networks for Thermal and NIR to Visible Image Transformation](https://mlsanity.com/paper/2002.07082/) (2020)
- [Matching Thermal to Visible Face Images Using a Semantic-Guided Generative Adversarial Network](https://mlsanity.com/paper/1903.00963/) (2019)
- [In2I : Unsupervised Multi-Image-to-Image Translation Using Generative Adversarial Networks](https://mlsanity.com/paper/1711.09334/) (2017)
- [Unsupervised Image-generation Enhanced Adaptation for Object Detection in Thermal images](https://mlsanity.com/paper/2002.06770/) (2020)
- [The Use of AI for Thermal Emotion Recognition: A Review of Problems and Limitations in Standard Design and Data](https://mlsanity.com/paper/2009.10589/) (2020)
- [Multi-Metric Evaluation of Thermal-to-Visual Face Recognition](https://mlsanity.com/paper/2007.11987/) (2020)

## Built on by (most cited first, 1 of 1)

- [When Visible-to-Thermal Facial GAN Beats Conditional Diffusion](https://mlsanity.com/paper/2302.09395/) (2023)

## Abstract (arXiv)

Thermal images reveal medically important physiological information about human stress, signs of inflammation, and emotional mood that cannot be seen on visible images. Providing a method to generate thermal faces from visible images would be highly valuable for the telemedicine community in order to show this medical information. To the best of our knowledge, there are limited works on visible-to-thermal (VT) face translation, and many current works go the opposite direction to generate visible faces from thermal surveillance images (TV) for law enforcement applications. As a result, we introduce favtGAN, a VT GAN which uses the pix2pix image translation model with an auxiliary sensor label prediction network for generating thermal faces from visible images. Since most TV methods are trained on only one data source drawn from one thermal sensor, we combine datasets from faces and cityscapes. These combined data are captured from similar sensors in order to bootstrap the training and transfer learning task, especially valuable because visible-thermal face datasets are limited. Experiments on these combined datasets show that favtGAN demonstrates an increase in SSIM and PSNR scores of generated thermal faces, compared to training on a single face dataset alone.