# Facial Feature Embedded CycleGAN for VIS-NIR Translation

arXiv: https://arxiv.org/abs/1904.09464 · HTML: https://mlsanity.com/paper/1904.09464/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Apr 2019.

## Built on by (most cited first, 4 of 4)

- [Spatial-Temporal Frequency Forgery Clue for Video Forgery Detection in VIS and NIR Scenario](https://mlsanity.com/paper/2207.01906/) (2022)
- [SASMU: boost the performance of generalized recognition model using synthetic face dataset](https://mlsanity.com/paper/2306.01449/) (2023)
- [Multi-Energy Guided Image Translation with Stochastic Differential Equations for Near-Infrared Facial Expression Recognition](https://mlsanity.com/paper/2312.05908/) (2023)
- [Unsupervised Network for Single Image Raindrop Removal](https://mlsanity.com/paper/2412.03019/) (2024)

## Abstract (arXiv)

VIS-NIR face recognition remains a challenging task due to the distinction between spectral components of two modalities and insufficient paired training data. Inspired by the CycleGAN, this paper presents a method aiming to translate VIS face images into fake NIR images whose distributions are intended to approximate those of true NIR images, which is achieved by proposing a new facial feature embedded CycleGAN. Firstly, to learn the particular feature of NIR domain while preserving common facial representation between VIS and NIR domains, we employ a general facial feature extractor (FFE) to replace the encoder in the original generator of CycleGAN. For implementing the facial feature extractor, herein the MobileFaceNet is pretrained on a VIS face database, and is able to extract effective features. Secondly, the domain-invariant feature learning is enhanced by considering a new pixel consistency loss. Lastly, we establish a new WHU VIS-NIR database which varies in face rotation and expressions to enrich the training data. Experimental results on the Oulu-CASIA NIR-VIS database and the WHU VIS-NIR database show that the proposed FFE-based CycleGAN (FFE-CycleGAN) outperforms state-of-the-art VIS-NIR face recognition methods and achieves 96.5% accuracy.