# Depth as Attention for Face Representation Learning

arXiv: https://arxiv.org/abs/2101.00652 · HTML: https://mlsanity.com/paper/2101.00652/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 82 references that are in the map.
- First posted on arXiv 3 Jan 2021.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [Bilinear Attention Networks](https://mlsanity.com/paper/1805.07932/) (2018)
- [Deep Face Recognition: A Survey](https://mlsanity.com/paper/1804.06655/) (2018)
- [Bilinear CNNs for Fine-grained Visual Recognition](https://mlsanity.com/paper/1504.07889/) (2015)
- [SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioning](https://mlsanity.com/paper/1611.05594/) (2016)
- [BAM: Bottleneck Attention Module](https://mlsanity.com/paper/1807.06514/) (2018)
- [An Attentive Survey of Attention Models](https://mlsanity.com/paper/1904.02874/) (2019)
- [Dual Attention Networks for Multimodal Reasoning and Matching](https://mlsanity.com/paper/1611.00471/) (2016)
- [Intel RealSense Stereoscopic Depth Cameras](https://mlsanity.com/paper/1705.05548/) (2017)
- [Generative Face Completion](https://mlsanity.com/paper/1704.05838/) (2017)
- [Deep Visual Attention Prediction](https://mlsanity.com/paper/1705.02544/) (2017)
- [Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering](https://mlsanity.com/paper/1511.05234/) (2015)
- [Occlusion Robust Face Recognition Based on Mask Learning with PairwiseDifferential Siamese Network](https://mlsanity.com/paper/1908.06290/) (2019)
- [Exploring Human-like Attention Supervision in Visual Question Answering](https://mlsanity.com/paper/1709.06308/) (2017)
- [When Face Recognition Meets with Deep Learning: an Evaluation of Convolutional Neural Networks for Face Recognition](https://mlsanity.com/paper/1504.02351/) (2015)
- [A Comprehensive Overview of Biometric Fusion](https://mlsanity.com/paper/1902.02919/) (2019)
- [Robust LSTM-Autoencoders for Face De-Occlusion in the Wild](https://mlsanity.com/paper/1612.08534/) (2016)
- [Ear Recognition: More Than a Survey](https://mlsanity.com/paper/1611.06203/) (2016)
- [Attention-Based Capsule Networks with Dynamic Routing for Relation Extraction](https://mlsanity.com/paper/1812.11321/) (2018)
- [Face Recognition: A Novel Multi-Level Taxonomy based Survey](https://mlsanity.com/paper/1901.00713/) (2019)
- [Robust RGB-D Face Recognition Using Attribute-Aware Loss](https://mlsanity.com/paper/1811.09847/) (2018)

## Built on by (most cited first, 5 of 5)

- [Local Multi-Head Channel Self-Attention for Facial Expression Recognition](https://mlsanity.com/paper/2111.07224/) (2021)
- [High-Accuracy RGB-D Face Recognition via Segmentation-Aware Face Depth Estimation and Mask-Guided Attention Network](https://mlsanity.com/paper/2112.11713/) (2021)
- [Teacher-Student Adversarial Depth Hallucination to Improve Face Recognition](https://mlsanity.com/paper/2104.02424/) (2021)
- [Privacy-Preserving Remote Heart Rate Estimation from Facial Videos](https://mlsanity.com/paper/2306.01141/) (2023)
- [Confidence-Aware RGB-D Face Recognition via Virtual Depth Synthesis](https://mlsanity.com/paper/2403.06529/) (2024)

## Abstract (arXiv)

Face representation learning solutions have recently achieved great success for various applications such as verification and identification. However, face recognition approaches that are based purely on RGB images rely solely on intensity information, and therefore are more sensitive to facial variations, notably pose, occlusions, and environmental changes such as illumination and background. A novel depth-guided attention mechanism is proposed for deep multi-modal face recognition using low-cost RGB-D sensors. Our novel attention mechanism directs the deep network "where to look" for visual features in the RGB image by focusing the attention of the network using depth features extracted by a Convolution Neural Network (CNN). The depth features help the network focus on regions of the face in the RGB image that contains more prominent person-specific information. Our attention mechanism then uses this correlation to generate an attention map for RGB images from the depth features extracted by CNN. We test our network on four public datasets, showing that the features obtained by our proposed solution yield better results on the Lock3DFace, CurtinFaces, IIIT-D RGB-D, and KaspAROV datasets which include challenging variations in pose, occlusion, illumination, expression, and time-lapse. Our solution achieves average (increased) accuracies of 87.3% (+5.0%), 99.1% (+0.9%), 99.7% (+0.6%) and 95.3%(+0.5%) for the four datasets respectively, thereby improving the state-of-the-art. We also perform additional experiments with thermal images, instead of depth images, showing the high generalization ability of our solution when adopting other modalities for guiding the attention mechanism instead of depth information