# Learning Domain and Pose Invariance for Thermal-to-Visible Face Recognition

arXiv: https://arxiv.org/abs/2211.09350 · HTML: https://mlsanity.com/paper/2211.09350/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 64 references that are in the map.
- First posted on arXiv 17 Nov 2022.

## Builds on (most cited first, 30 of 33)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Learning Face Representation from Scratch](https://mlsanity.com/paper/1411.7923/) (2014)
- [The MegaFace Benchmark: 1 Million Faces for Recognition at Scale](https://mlsanity.com/paper/1512.00596/) (2015)
- [Deep Learning Face Representation by Joint Identification-Verification](https://mlsanity.com/paper/1406.4773/) (2014)
- [Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis](https://mlsanity.com/paper/1704.04086/) (2017)
- [Towards Large-Pose Face Frontalization in the Wild](https://mlsanity.com/paper/1704.06244/) (2017)
- [Level Playing Field for Million Scale Face Recognition](https://mlsanity.com/paper/1705.00393/) (2017)
- [Pose-Robust Face Recognition via Deep Residual Equivariant Mapping](https://mlsanity.com/paper/1803.00839/) (2018)
- [Dual Variational Generation for Low-Shot Heterogeneous Face Recognition](https://mlsanity.com/paper/1903.10203/) (2019)
- [Synthesis of High-Quality Visible Faces from Polarimetric Thermal Faces using Generative Adversarial Networks](https://mlsanity.com/paper/1812.05155/) (2018)
- [A Large-Scale, Time-Synchronized Visible and Thermal Face Dataset](https://mlsanity.com/paper/2101.02637/) (2021)
- [Not Afraid of the Dark: NIR-VIS Face Recognition via Cross-spectral Hallucination and Low-rank Embedding](https://mlsanity.com/paper/1611.06638/) (2016)
- [Polarimetric Thermal to Visible Face Verification via Self-Attention Guided Synthesis](https://mlsanity.com/paper/1904.07344/) (2019)
- [Do We Really Need to Collect Millions of Faces for Effective Face Recognition?](https://mlsanity.com/paper/1603.07057/) (2016)
- [Cross-Domain Identification for Thermal-to-Visible Face Recognition](https://mlsanity.com/paper/2008.08473/) (2020)
- [Dual-Attention GAN for Large-Pose Face Frontalization](https://mlsanity.com/paper/2002.07227/) (2020)
- [Generative Adversarial Network-based Synthesis of Visible Faces from Polarimetric Thermal Faces](https://mlsanity.com/paper/1708.02681/) (2017)
- [Learning a High Fidelity Pose Invariant Model for High-resolution Face Frontalization](https://mlsanity.com/paper/1806.08472/) (2018)
- [Multi-Scale Thermal to Visible Face Verification via Attribute Guided Synthesis](https://mlsanity.com/paper/2004.09502/) (2020)
- [Shared Representation Learning for Heterogeneous Face Recognition](https://mlsanity.com/paper/1406.1247/) (2014)
- [Thermal to Visible Synthesis of Face Images using Multiple Regions](https://mlsanity.com/paper/1803.07599/) (2018)
- [Polarimetric Thermal to Visible Face Verification via Attribute Preserved Synthesis](https://mlsanity.com/paper/1901.00889/) (2019)
- [Simultaneous Face Hallucination and Translation for Thermal to Visible Face Verification using Axial-GAN](https://mlsanity.com/paper/2104.06534/) (2021)
- [A Synthesis-Based Approach for Thermal-to-Visible Face Verification](https://mlsanity.com/paper/2108.09558/) (2021)
- [Deep Perceptual Mapping for Cross-Modal Face Recognition](https://mlsanity.com/paper/1601.05347/) (2016)
- [VIPLFaceNet: An Open Source Deep Face Recognition SDK](https://mlsanity.com/paper/1609.03892/) (2016)

## Built on by (most cited first, 2 of 2)

- [2D-3D Attention and Entropy for Pose Robust 2D Facial Recognition](https://mlsanity.com/paper/2505.09073/) (2025)
- [Cross-Spectral Attention for Unsupervised RGB-IR Face Verification and Person Re-identification](https://mlsanity.com/paper/2411.19215/) (2024)

## Abstract (arXiv)

Interest in thermal to visible face recognition has grown significantly over the last decade due to advancements in thermal infrared cameras and analytics beyond the visible spectrum. Despite large discrepancies between thermal and visible spectra, existing approaches bridge domain gaps by either synthesizing visible faces from thermal faces or by learning the cross-spectrum image representations. These approaches typically work well with frontal facial imagery collected at varying ranges and expressions, but exhibit significantly reduced performance when matching thermal faces with varying poses to frontal visible faces. We propose a novel Domain and Pose Invariant Framework that simultaneously learns domain and pose invariant representations. Our proposed framework is composed of modified networks for extracting the most correlated intermediate representations from off-pose thermal and frontal visible face imagery, a sub-network to jointly bridge domain and pose gaps, and a joint-loss function comprised of cross-spectrum and pose-correction losses. We demonstrate efficacy and advantages of the proposed method by evaluating on three thermal-visible datasets: ARL Visible-to-Thermal Face, ARL Multimodal Face, and Tufts Face. Although DPIF focuses on learning to match off-pose thermal to frontal visible faces, we also show that DPIF enhances performance when matching frontal thermal face images to frontal visible face images.