# High-Accuracy RGB-D Face Recognition via Segmentation-Aware Face Depth Estimation and Mask-Guided Attention Network

arXiv: https://arxiv.org/abs/2112.11713 · HTML: https://mlsanity.com/paper/2112.11713/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 44 references that are in the map.
- First posted on arXiv 22 Dec 2021.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [Learning Face Representation from Scratch](https://mlsanity.com/paper/1411.7923/) (2014)
- [Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network](https://mlsanity.com/paper/1803.07835/) (2018)
- [Face Alignment in Full Pose Range: A 3D Total Solution](https://mlsanity.com/paper/1804.01005/) (2018)
- [Cross Modal Focal Loss for RGBD Face Anti-Spoofing](https://mlsanity.com/paper/2103.00948/) (2021)
- [Improving the Performance of Unimodal Dynamic Hand-Gesture Recognition with Multimodal Training](https://mlsanity.com/paper/1812.06145/) (2018)
- [Learning from Millions of 3D Scans for Large-scale 3D Face Recognition](https://mlsanity.com/paper/1711.05942/) (2017)
- [Deep 3D Face Identification](https://mlsanity.com/paper/1703.10714/) (2017)
- [Robust RGB-D Face Recognition Using Attribute-Aware Loss](https://mlsanity.com/paper/1811.09847/) (2018)
- [Depth as Attention for Face Representation Learning](https://mlsanity.com/paper/2101.00652/) (2021)
- [Data-Free Point Cloud Network for 3D Face Recognition](https://mlsanity.com/paper/1911.04731/) (2019)
- [Two-Level Attention-based Fusion Learning for RGB-D Face Recognition](https://mlsanity.com/paper/2003.00168/) (2020)

## Built on by (most cited first, 5 of 5)

- [RetouchingFFHQ: A Large-scale Dataset for Fine-grained Face Retouching Detection](https://mlsanity.com/paper/2307.10642/) (2023)
- [Exploring Depth Information for Detecting Manipulated Face Videos](https://mlsanity.com/paper/2411.18572/) (2024)
- [Exploring Depth Information for Face Manipulation Detection](https://mlsanity.com/paper/2212.14230/) (2022)
- [Improving 2D face recognition via fine-level facial depth generation and RGB-D complementary feature learning](https://mlsanity.com/paper/2305.04426/) (2023)
- [Confidence-Aware RGB-D Face Recognition via Virtual Depth Synthesis](https://mlsanity.com/paper/2403.06529/) (2024)

## Abstract (arXiv)

Deep learning approaches have achieved highly accurate face recognition by training the models with very large face image datasets. Unlike the availability of large 2D face image datasets, there is a lack of large 3D face datasets available to the public. Existing public 3D face datasets were usually collected with few subjects, leading to the over-fitting problem. This paper proposes two CNN models to improve the RGB-D face recognition task. The first is a segmentation-aware depth estimation network, called DepthNet, which estimates depth maps from RGB face images by including semantic segmentation information for more accurate face region localization. The other is a novel mask-guided RGB-D face recognition model that contains an RGB recognition branch, a depth map recognition branch, and an auxiliary segmentation mask branch with a spatial attention module. Our DepthNet is used to augment a large 2D face image dataset to a large RGB-D face dataset, which is used for training an accurate RGB-D face recognition model. Furthermore, the proposed mask-guided RGB-D face recognition model can fully exploit the depth map and segmentation mask information and is more robust against pose variation than previous methods. Our experimental results show that DepthNet can produce more reliable depth maps from face images with the segmentation mask. Our mask-guided face recognition model outperforms state-of-the-art methods on several public 3D face datasets.