# Heterogeneous Face Recognition via Face Synthesis with Identity-Attribute Disentanglement

arXiv: https://arxiv.org/abs/2206.04854 · HTML: https://mlsanity.com/paper/2206.04854/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 89 references that are in the map.
- First posted on arXiv 10 Jun 2022.

## Builds on (most cited first, 30 of 30)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [Deep Face Recognition: A Survey](https://mlsanity.com/paper/1804.06655/) (2018)
- [A Light CNN for Deep Face Representation with Noisy Labels](https://mlsanity.com/paper/1511.02683/) (2015)
- [On Face Segmentation, Face Swapping, and Face Perception](https://mlsanity.com/paper/1704.06729/) (2017)
- [Wasserstein CNN: Learning Invariant Features for NIR-VIS Face Recognition](https://mlsanity.com/paper/1708.02412/) (2017)
- [DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition](https://mlsanity.com/paper/2009.09399/) (2020)
- [Dual Variational Generation for Low-Shot Heterogeneous Face Recognition](https://mlsanity.com/paper/1903.10203/) (2019)
- [Synthesis of High-Quality Visible Faces from Polarimetric Thermal Faces using Generative Adversarial Networks](https://mlsanity.com/paper/1812.05155/) (2018)
- [Soft-IntroVAE: Analyzing and Improving the Introspective Variational Autoencoder](https://mlsanity.com/paper/2012.13253/) (2020)
- [Not Afraid of the Dark: NIR-VIS Face Recognition via Cross-spectral Hallucination and Low-rank Embedding](https://mlsanity.com/paper/1611.06638/) (2016)
- [Training Deep Face Recognition Systems with Synthetic Data](https://mlsanity.com/paper/1802.05891/) (2018)
- [AOT: Appearance Optimal Transport Based Identity Swapping for Forgery Detection](https://mlsanity.com/paper/2011.02674/) (2020)
- [Disentangled Variational Representation for Heterogeneous Face Recognition](https://mlsanity.com/paper/1809.01936/) (2018)
- [Cross-Spectral Face Hallucination via Disentangling Independent Factors](https://mlsanity.com/paper/1909.04365/) (2019)
- [Adversarial Discriminative Heterogeneous Face Recognition](https://mlsanity.com/paper/1709.03675/) (2017)
- [Coupled Deep Learning for Heterogeneous Face Recognition](https://mlsanity.com/paper/1704.02450/) (2017)
- [Shared Representation Learning for Heterogeneous Face Recognition](https://mlsanity.com/paper/1406.1247/) (2014)
- [LAMP-HQ: A Large-Scale Multi-Pose High-Quality Database and Benchmark for NIR-VIS Face Recognition](https://mlsanity.com/paper/1912.07809/) (2019)
- [Relational Deep Feature Learning for Heterogeneous Face Recognition](https://mlsanity.com/paper/2003.00697/) (2020)
- [Facial Synthesis from Visual Attributes via Sketch using Multi-Scale Generators](https://mlsanity.com/paper/1912.10479/) (2019)
- [Thermal to Visible Face Recognition Using Deep Autoencoders](https://mlsanity.com/paper/2002.04219/) (2020)
- [Multimodal Face Synthesis from Visual Attributes](https://mlsanity.com/paper/2104.04362/) (2021)

## Built on by (most cited first, 4 of 4)

- [FPGAN-Control: A Controllable Fingerprint Generator for Training with Synthetic Data](https://mlsanity.com/paper/2310.19024/) (2023)
- [Precise Facial Landmark Detection by Dynamic Semantic Aggregation Transformer](https://mlsanity.com/paper/2412.00740/) (2024)
- [IFAST: Weakly Supervised Interpretable Face Anti-spoofing from Single-shot Binocular NIR Images](https://mlsanity.com/paper/2309.17399/) (2023)
- [FVeinSyn: Synthetic Finger Vein Image Generator](https://mlsanity.com/paper/2608.27527/) (2026)

## Abstract (arXiv)

Heterogeneous Face Recognition (HFR) aims to match faces across different domains (e.g., visible to near-infrared images), which has been widely applied in authentication and forensics scenarios. However, HFR is a challenging problem because of the large cross-domain discrepancy, limited heterogeneous data pairs, and large variation of facial attributes. To address these challenges, we propose a new HFR method from the perspective of heterogeneous data augmentation, named Face Synthesis with Identity-Attribute Disentanglement (FSIAD). Firstly, the identity-attribute disentanglement (IAD) decouples face images into identity-related representations and identity-unrelated representations (called attributes), and then decreases the correlation between identities and attributes. Secondly, we devise a face synthesis module (FSM) to generate a large number of images with stochastic combinations of disentangled identities and attributes for enriching the attribute diversity of synthetic images. Both the original images and the synthetic ones are utilized to train the HFR network for tackling the challenges and improving the performance of HFR. Extensive experiments on five HFR databases validate that FSIAD obtains superior performance than previous HFR approaches. Particularly, FSIAD obtains 4.8% improvement over state of the art in terms of VR@FAR=0.01% on LAMP-HQ, the largest HFR database so far.