# Confidence-Aware RGB-D Face Recognition via Virtual Depth Synthesis

arXiv: https://arxiv.org/abs/2403.06529 · HTML: https://mlsanity.com/paper/2403.06529/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 29 references that are in the map.
- First posted on arXiv 11 Mar 2024.

## Builds on (most cited first, 9 of 9)

- [Learning Face Representation from Scratch](https://mlsanity.com/paper/1411.7923/) (2014)
- [Learning Confidence for Out-of-Distribution Detection in Neural Networks](https://mlsanity.com/paper/1802.04865/) (2018)
- [MobileFaceNets: Efficient CNNs for Accurate Real-Time Face Verification on Mobile Devices](https://mlsanity.com/paper/1804.07573/) (2018)
- [Morphable Face Models - An Open Framework](https://mlsanity.com/paper/1709.08398/) (2017)
- [Learning from Millions of 3D Scans for Large-scale 3D Face Recognition](https://mlsanity.com/paper/1711.05942/) (2017)
- [Face Recognition: A Novel Multi-Level Taxonomy based Survey](https://mlsanity.com/paper/1901.00713/) (2019)
- [Depth as Attention for Face Representation Learning](https://mlsanity.com/paper/2101.00652/) (2021)
- [High-Accuracy RGB-D Face Recognition via Segmentation-Aware Face Depth Estimation and Mask-Guided Attention Network](https://mlsanity.com/paper/2112.11713/) (2021)
- [Teacher-Student Adversarial Depth Hallucination to Improve Face Recognition](https://mlsanity.com/paper/2104.02424/) (2021)

## Abstract (arXiv)

2D face recognition encounters challenges in unconstrained environments due to varying illumination, occlusion, and pose. Recent studies focus on RGB-D face recognition to improve robustness by incorporating depth information. However, collecting sufficient paired RGB-D training data is expensive and time-consuming, hindering wide deployment. In this work, we first construct a diverse depth dataset generated by 3D Morphable Models for depth model pre-training. Then, we propose a domain-independent pre-training framework that utilizes readily available pre-trained RGB and depth models to separately perform face recognition without needing additional paired data for retraining. To seamlessly integrate the two distinct networks and harness the complementary benefits of RGB and depth information for improved accuracy, we propose an innovative Adaptive Confidence Weighting (ACW). This mechanism is designed to learn confidence estimates for each modality to achieve modality fusion at the score level. Our method is simple and lightweight, only requiring ACW training beyond the backbone models. Experiments on multiple public RGB-D face recognition benchmarks demonstrate state-of-the-art performance surpassing previous methods based on depth estimation and feature fusion, validating the efficacy of our approach.