# On the Effect of Selfie Beautification Filters on Face Detection and Recognition

arXiv: https://arxiv.org/abs/2110.08934 · HTML: https://mlsanity.com/paper/2110.08934/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 51 references that are in the map.
- First posted on arXiv 17 Oct 2021.

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [A survey of face recognition techniques under occlusion](https://mlsanity.com/paper/2006.11366/) (2020)
- [The Effect of Wearing a Mask on Face Recognition Performance: an Exploratory Study](https://mlsanity.com/paper/2007.13521/) (2020)
- [A Survey on Periocular Biometrics Research](https://mlsanity.com/paper/1810.03360/) (2018)
- [SqueezeFacePoseNet: Lightweight Face Verification Across Different Poses for Mobile Platforms](https://mlsanity.com/paper/2007.08566/) (2020)
- [Demography-based Facial Retouching Detection using Subclass Supervised Sparse Autoencoder](https://mlsanity.com/paper/1709.07598/) (2017)
- [On Detecting GANs and Retouching based Synthetic Alterations](https://mlsanity.com/paper/1901.09237/) (2019)

## Built on by (most cited first, 10 of 10)

- [Deep Learning for Iris Recognition: A Survey](https://mlsanity.com/paper/2210.05866/) (2022)
- [HumanVLM: Foundation for Human-Scene Vision-Language Model](https://mlsanity.com/paper/2411.03034/) (2024)
- [Diffusion-based Facial Aesthetics Enhancement with 3D Structure Guidance](https://mlsanity.com/paper/2503.14402/) (2025)
- [LFW-Beautified: A Dataset of Face Images with Beautification and Augmented Reality Filters](https://mlsanity.com/paper/2203.06082/) (2022)
- [OpenFilter: A Framework to Democratize Research Access to Social Media AR Filters](https://mlsanity.com/paper/2207.12319/) (2022)
- [IMAGINE: An Integrated Model of Artificial Intelligence-Mediated Communication Effects](https://mlsanity.com/paper/2212.08658/) (2022)
- [Deceptive Beauty: Evaluating the Impact of Beauty Filters on Deepfake and Morphing Attack Detection](https://mlsanity.com/paper/2509.14120/) (2025)
- [Exploring Complementarity and Explainability in CNNs for Periocular Verification Across Acquisition Distances](https://mlsanity.com/paper/2510.26282/) (2025)
- [Leveraging Large-Scale Face Datasets for Deep Periocular Recognition via Ocular Cropping](https://mlsanity.com/paper/2510.26294/) (2025)
- [Periocular Soft Biometrics: A Survey and Applications to Multimedia Forensics and Disinformation Detection](https://mlsanity.com/paper/2608.14701/) (2026)

## Abstract (arXiv)

Beautification and augmented reality filters are very popular in applications that use selfie images captured with smartphones or personal devices. However, they can distort or modify biometric features, severely affecting the capability of recognizing individuals' identity or even detecting the face. Accordingly, we address the effect of such filters on the accuracy of automated face detection and recognition. The social media image filters studied either modify the image contrast or illumination or occlude parts of the face with for example artificial glasses or animal noses. We observe that the effect of some of these filters is harmful both to face detection and identity recognition, specially if they obfuscate the eye or (to a lesser extent) the nose. To counteract such effect, we develop a method to reconstruct the applied manipulation with a modified version of the U-NET segmentation network. This is observed to contribute to a better face detection and recognition accuracy. From a recognition perspective, we employ distance measures and trained machine learning algorithms applied to features extracted using a ResNet-34 network trained to recognize faces. We also evaluate if incorporating filtered images to the training set of machine learning approaches are beneficial for identity recognition. Our results show good recognition when filters do not occlude important landmarks, specially the eyes (identification accuracy >99%, EER<2%). The combined effect of the proposed approaches also allow to mitigate the effect produced by filters that occlude parts of the face, achieving an identification accuracy of >92% with the majority of perturbations evaluated, and an EER <8%. Although there is room for improvement, when neither U-NET reconstruction nor training with filtered images is applied, the accuracy with filters that severely occlude the eye is <72% (identification) and >12% (EER)