# Masked Faces with Faced Masks

arXiv: https://arxiv.org/abs/2201.06427 · HTML: https://mlsanity.com/paper/2201.06427/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 53 references that are in the map.
- First posted on arXiv 17 Jan 2022.

## Builds on (most cited first, 23 of 23)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [Improving Transferability of Adversarial Examples with Input Diversity](https://mlsanity.com/paper/1803.06978/) (2018)
- [Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks](https://mlsanity.com/paper/1904.02884/) (2019)
- [Large-Margin Softmax Loss for Convolutional Neural Networks](https://mlsanity.com/paper/1612.02295/) (2016)
- [Additive Margin Softmax for Face Verification](https://mlsanity.com/paper/1801.05599/) (2018)
- [The MegaFace Benchmark: 1 Million Faces for Recognition at Scale](https://mlsanity.com/paper/1512.00596/) (2015)
- [AdvHat: Real-world adversarial attack on ArcFace Face ID system](https://mlsanity.com/paper/1908.08705/) (2019)
- [Range Loss for Deep Face Recognition with Long-tail](https://mlsanity.com/paper/1611.08976/) (2016)
- [Adv-Makeup: A New Imperceptible and Transferable Attack on Face Recognition](https://mlsanity.com/paper/2105.03162/) (2021)
- [Masked Face Recognition Dataset and Application](https://mlsanity.com/paper/2003.09093/) (2020)
- [Self-restrained Triplet Loss for Accurate Masked Face Recognition](https://mlsanity.com/paper/2103.01716/) (2021)
- [AVA: Adversarial Vignetting Attack against Visual Recognition](https://mlsanity.com/paper/2105.05558/) (2021)
- [MFR 2021: Masked Face Recognition Competition](https://mlsanity.com/paper/2106.15288/) (2021)
- [Amora: Black-box Adversarial Morphing Attack](https://mlsanity.com/paper/1912.03829/) (2019)
- [Adversarial Relighting Against Face Recognition](https://mlsanity.com/paper/2108.07920/) (2021)
- [Learning to Adversarially Blur Visual Object Tracking](https://mlsanity.com/paper/2107.12085/) (2021)
- [Boosting Masked Face Recognition with Multi-Task ArcFace](https://mlsanity.com/paper/2104.09874/) (2021)

## Built on by (most cited first, 2 of 2)

- [Towards Effective Adversarial Textured 3D Meshes on Physical Face Recognition](https://mlsanity.com/paper/2303.15818/) (2023)
- [Controllable Evaluation and Generation of Physical Adversarial Patch on Face Recognition](https://mlsanity.com/paper/2203.04623/) (2022)

## Abstract (arXiv)

Modern face recognition systems (FRS) still fall short when the subjects are wearing facial masks, a common theme in the age of respiratory pandemics. An intuitive partial remedy is to add a mask detector to flag any masked faces so that the FRS can act accordingly for those low-confidence masked faces. In this work, we set out to investigate the potential vulnerability of such FRS equipped with a mask detector, on large-scale masked faces, which might trigger a serious risk, e.g., letting a suspect evade the FRS where both facial identity and mask are undetected. As existing face recognizers and mask detectors have high performance in their respective tasks, it is significantly challenging to simultaneously fool them and preserve the transferability of the attack. We formulate the new task as the generation of realistic & adversarial-faced mask and make three main contributions: First, we study the naive Delanunay-based masking method (DM) to simulate the process of wearing a faced mask that is cropped from a template image, which reveals the main challenges of this new task. Second, we further equip the DM with the adversarial noise attack and propose the adversarial noise Delaunay-based masking method (AdvNoise-DM) that can fool the face recognition and mask detection effectively but make the face less natural. Third, we propose the adversarial filtering Delaunay-based masking method denoted as MF2M by employing the adversarial filtering for AdvNoise-DM and obtain more natural faces. With the above efforts, the final version not only leads to significant performance deterioration of the state-of-the-art (SOTA) deep learning-based FRS, but also remains undetected by the SOTA facial mask detector, thus successfully fooling both systems at the same time.