# Disentangled Variational Representation for Heterogeneous Face Recognition

arXiv: https://arxiv.org/abs/1809.01936 · HTML: https://mlsanity.com/paper/1809.01936/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2018.

## Built on by (most cited first, 13 of 13)

- [Learning to Refactor Action and Co-occurrence Features for Temporal Action Localization](https://mlsanity.com/paper/2206.11493/) (2022)
- [Prepended Domain Transformer: Heterogeneous Face Recognition without Bells and Whistles](https://mlsanity.com/paper/2210.06529/) (2022)
- [Simultaneous Face Hallucination and Translation for Thermal to Visible Face Verification using Axial-GAN](https://mlsanity.com/paper/2104.06534/) (2021)
- [PoseFace: Pose-Invariant Features and Pose-Adaptive Loss for Face Recognition](https://mlsanity.com/paper/2107.11721/) (2021)
- [CrossDF: Improving Cross-Domain Deepfake Detection with Deep Information Decomposition](https://mlsanity.com/paper/2310.00359/) (2023)
- [xEdgeFace: Efficient Cross-Spectral Face Recognition for Edge Devices](https://mlsanity.com/paper/2504.19646/) (2025)
- [Heterogeneous Face Recognition via Face Synthesis with Identity-Attribute Disentanglement](https://mlsanity.com/paper/2206.04854/) (2022)
- [DAIL: Dataset-Aware and Invariant Learning for Face Recognition](https://mlsanity.com/paper/2101.05419/) (2021)
- [Physically-Based Face Rendering for NIR-VIS Face Recognition](https://mlsanity.com/paper/2211.06408/) (2022)
- [From Modalities to Styles: Rethinking the Domain Gap in Heterogeneous Face Recognition](https://mlsanity.com/paper/2404.14247/) (2024)
- [Multimodal Face Synthesis from Visual Attributes](https://mlsanity.com/paper/2104.04362/) (2021)
- [Joint Feature Distribution Alignment Learning for NIR-VIS and VIS-VIS Face Recognition](https://mlsanity.com/paper/2204.11434/) (2022)
- [Lightweight Cross-Spectral Face Recognition via Contrastive Alignment and Distillation](https://mlsanity.com/paper/2605.04769/) (2026)

## Abstract (arXiv)

Visible (VIS) to near infrared (NIR) face matching is a challenging problem due to the significant domain discrepancy between the domains and a lack of sufficient data for training cross-modal matching algorithms. Existing approaches attempt to tackle this problem by either synthesizing visible faces from NIR faces, extracting domain-invariant features from these modalities, or projecting heterogeneous data onto a common latent space for cross-modal matching. In this paper, we take a different approach in which we make use of the Disentangled Variational Representation (DVR) for cross-modal matching. First, we model a face representation with an intrinsic identity information and its within-person variations. By exploring the disentangled latent variable space, a variational lower bound is employed to optimize the approximate posterior for NIR and VIS representations. Second, aiming at obtaining more compact and discriminative disentangled latent space, we impose a minimization of the identity information for the same subject and a relaxed correlation alignment constraint between the NIR and VIS modality variations. An alternative optimization scheme is proposed for the disentangled variational representation part and the heterogeneous face recognition network part. The mutual promotion between these two parts effectively reduces the NIR and VIS domain discrepancy and alleviates over-fitting. Extensive experiments on three challenging NIR-VIS heterogeneous face recognition databases demonstrate that the proposed method achieves significant improvements over the state-of-the-art methods.