# Multi-Dataset Benchmarks for Masked Identification using Contrastive Representation Learning

arXiv: https://arxiv.org/abs/2106.05596 · HTML: https://mlsanity.com/paper/2106.05596/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 44 references that are in the map.
- First posted on arXiv 10 Jun 2021.

## Builds on (most cited first, 19 of 19)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks](https://mlsanity.com/paper/1406.6909/) (2014)
- [Representation Learning by Learning to Count](https://mlsanity.com/paper/1708.06734/) (2017)
- [Masked Face Recognition Dataset and Application](https://mlsanity.com/paper/2003.09093/) (2020)
- [Occlusion Robust Face Recognition Based on Mask Learning with PairwiseDifferential Siamese Network](https://mlsanity.com/paper/1908.06290/) (2019)
- [The Effect of Wearing a Mask on Face Recognition Performance: an Exploratory Study](https://mlsanity.com/paper/2007.13521/) (2020)
- [Efficient Masked Face Recognition Method during the COVID-19 Pandemic](https://mlsanity.com/paper/2105.03026/) (2021)
- [Boosting Masked Face Recognition with Multi-Task ArcFace](https://mlsanity.com/paper/2104.09874/) (2021)
- [AFIF4: Deep Gender Classification based on AdaBoost-based Fusion of Isolated Facial Features and Foggy Faces](https://mlsanity.com/paper/1706.04277/) (2017)
- [Masked Face Recognition: Human vs. Machine](https://mlsanity.com/paper/2103.01924/) (2021)
- [Towards NIR-VIS Masked Face Recognition](https://mlsanity.com/paper/2104.06761/) (2021)
- [Masked Face Recognition using ResNet-50](https://mlsanity.com/paper/2104.08997/) (2021)

## Built on by (most cited first, 3 of 3)

- [Synthetic Data in Human Analysis: A Survey](https://mlsanity.com/paper/2208.09191/) (2022)
- [Does a Face Mask Protect my Privacy?: Deep Learning to Predict Protected Attributes from Masked Face Images](https://mlsanity.com/paper/2112.07879/) (2021)
- [An Exploratory Study of Masked Face Recognition with Machine Learning Algorithms](https://mlsanity.com/paper/2306.08549/) (2023)

## Abstract (arXiv)

The COVID-19 pandemic has drastically changed accepted norms globally. Within the past year, masks have been used as a public health response to limit the spread of the virus. This sudden change has rendered many face recognition based access control, authentication and surveillance systems ineffective. Official documents such as passports, driving license and national identity cards are enrolled with fully uncovered face images. However, in the current global situation, face matching systems should be able to match these reference images with masked face images. As an example, in an airport or security checkpoint it is safer to match the unmasked image of the identifying document to the masked person rather than asking them to remove the mask. We find that current facial recognition techniques are not robust to this form of occlusion. To address this unique requirement presented due to the current circumstance, we propose a set of re-purposed datasets and a benchmark for researchers to use. We also propose a contrastive visual representation learning based pre-training workflow which is specialized to masked vs unmasked face matching. We ensure that our method learns robust features to differentiate people across varying data collection scenarios. We achieve this by training over many different datasets and validating our result by testing on various holdout datasets. The specialized weights trained by our method outperform standard face recognition features for masked to unmasked face matching. We believe the provided synthetic mask generating code, our novel training approach and the trained weights from the masked face models will help in adopting existing face recognition systems to operate in the current global environment. We open-source all contributions for broader use by the research community.