# Not Afraid of the Dark: NIR-VIS Face Recognition via Cross-spectral Hallucination and Low-rank Embedding

arXiv: https://arxiv.org/abs/1611.06638 · HTML: https://mlsanity.com/paper/1611.06638/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Nov 2016.

## Built on by (most cited first, 22 of 22)

- [Deep Learning-based Face Super-Resolution: A Survey](https://mlsanity.com/paper/2101.03749/) (2021)
- [Prepended Domain Transformer: Heterogeneous Face Recognition without Bells and Whistles](https://mlsanity.com/paper/2210.06529/) (2022)
- [Visible-Infrared Person Re-Identification Using Privileged Intermediate Information](https://mlsanity.com/paper/2209.09348/) (2022)
- [Robust 3D Object Detection from LiDAR-Radar Point Clouds via Cross-Modal Feature Augmentation](https://mlsanity.com/paper/2309.17336/) (2023)
- [Cross-spectral Gated-RGB Stereo Depth Estimation](https://mlsanity.com/paper/2405.12759/) (2024)
- [Beyond the Visible: A Survey on Cross-spectral Face Recognition](https://mlsanity.com/paper/2201.04435/) (2022)
- [Adaptive Generation of Privileged Intermediate Information for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2307.03240/) (2023)
- [Recent Advances in Deep Learning Techniques for Face Recognition](https://mlsanity.com/paper/2103.10492/) (2021)
- [Towards NIR-VIS Masked Face Recognition](https://mlsanity.com/paper/2104.06761/) (2021)
- [Unlocking Visual Secrets: Inverting Features with Diffusion Priors for Image Reconstruction](https://mlsanity.com/paper/2412.10448/) (2024)
- [xEdgeFace: Efficient Cross-Spectral Face Recognition for Edge Devices](https://mlsanity.com/paper/2504.19646/) (2025)
- [Heterogeneous Face Recognition via Face Synthesis with Identity-Attribute Disentanglement](https://mlsanity.com/paper/2206.04854/) (2022)
- [Physically-Based Face Rendering for NIR-VIS Face Recognition](https://mlsanity.com/paper/2211.06408/) (2022)
- [From Modalities to Styles: Rethinking the Domain Gap in Heterogeneous Face Recognition](https://mlsanity.com/paper/2404.14247/) (2024)
- [Template-based Multi-Domain Face Recognition](https://mlsanity.com/paper/2409.09832/) (2024)
- [Learning Domain and Pose Invariance for Thermal-to-Visible Face Recognition](https://mlsanity.com/paper/2211.09350/) (2022)
- [A NIR-to-VIS face recognition via part adaptive and relation attention module](https://mlsanity.com/paper/2102.00689/) (2021)
- [A Survey of Face Recognition](https://mlsanity.com/paper/2212.13038/) (2022)
- [Lightweight Cross-Spectral Face Recognition via Contrastive Alignment and Distillation](https://mlsanity.com/paper/2605.04769/) (2026)
- [Understanding Cross Domain Presentation Attack Detection for Visible Face Recognition](https://mlsanity.com/paper/2111.02548/) (2021)
- [Escaping Data Scarcity for High-Resolution Heterogeneous Face Hallucination](https://mlsanity.com/paper/2203.16669/) (2022)
- [NIR-to-VIS Face Recognition via Embedding Relations and Coordinates of the Pairwise Features](https://mlsanity.com/paper/2208.02417/) (2022)

## Abstract (arXiv)

Surveillance cameras today often capture NIR (near infrared) images in low-light environments. However, most face datasets accessible for training and verification are only collected in the VIS (visible light) spectrum. It remains a challenging problem to match NIR to VIS face images due to the different light spectrum. Recently, breakthroughs have been made for VIS face recognition by applying deep learning on a huge amount of labeled VIS face samples. The same deep learning approach cannot be simply applied to NIR face recognition for two main reasons: First, much limited NIR face images are available for training compared to the VIS spectrum. Second, face galleries to be matched are mostly available only in the VIS spectrum. In this paper, we propose an approach to extend the deep learning breakthrough for VIS face recognition to the NIR spectrum, without retraining the underlying deep models that see only VIS faces. Our approach consists of two core components, cross-spectral hallucination and low-rank embedding, to optimize respectively input and output of a VIS deep model for cross-spectral face recognition. Cross-spectral hallucination produces VIS faces from NIR images through a deep learning approach. Low-rank embedding restores a low-rank structure for faces deep features across both NIR and VIS spectrum. We observe that it is often equally effective to perform hallucination to input NIR images or low-rank embedding to output deep features for a VIS deep model for cross-spectral recognition. When hallucination and low-rank embedding are deployed together, we observe significant further improvement; we obtain state-of-the-art accuracy on the CASIA NIR-VIS v2.0 benchmark, without the need at all to re-train the recognition system.