# Explainable AI for Analyzing Person-Specific Patterns in Facial Recognition Tasks

arXiv: https://arxiv.org/abs/2509.17457 · HTML: https://mlsanity.com/paper/2509.17457/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 48 references that are in the map.
- First posted on arXiv 22 Sep 2025.

## Builds on (most cited first, 19 of 19)

- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [Learning Face Representation from Scratch](https://mlsanity.com/paper/1411.7923/) (2014)
- [Invertible Residual Networks](https://mlsanity.com/paper/1811.00995/) (2018)
- [Efficient Decision-based Black-box Adversarial Attacks on Face Recognition](https://mlsanity.com/paper/1904.04433/) (2019)
- [Partial FC: Training 10 Million Identities on a Single Machine](https://mlsanity.com/paper/2010.05222/) (2020)
- [LowKey: Leveraging Adversarial Attacks to Protect Social Media Users from Facial Recognition](https://mlsanity.com/paper/2101.07922/) (2021)
- [Is Cosine-Similarity of Embeddings Really About Similarity?](https://mlsanity.com/paper/2403.05440/) (2024)
- [A General Framework for Adversarial Examples with Objectives](https://mlsanity.com/paper/1801.00349/) (2017)
- [The Devil of Face Recognition is in the Noise](https://mlsanity.com/paper/1807.11649/) (2018)
- [Towards Effective Adversarial Textured 3D Meshes on Physical Face Recognition](https://mlsanity.com/paper/2303.15818/) (2023)
- [Adapting Grad-CAM for Embedding Networks](https://mlsanity.com/paper/2001.06538/) (2020)
- [Dodging Attack Using Carefully Crafted Natural Makeup](https://mlsanity.com/paper/2109.06467/) (2021)
- [Controllable Evaluation and Generation of Physical Adversarial Patch on Face Recognition](https://mlsanity.com/paper/2203.04623/) (2022)
- [The Hidden Pitfalls of the Cosine Similarity Loss](https://mlsanity.com/paper/2406.16468/) (2024)
- [Generalizing GradCAM for Embedding Networks](https://mlsanity.com/paper/2402.00909/) (2024)

## Built on by (most cited first, 1 of 1)

- [Adversarial Camouflage](https://mlsanity.com/paper/2603.21867/) (2026)

## Abstract (arXiv)

The proliferation of facial recognition systems presents major privacy risks, driving the need for effective countermeasures. Current adversarial techniques apply generalized methods rather than adapting to individual facial characteristics, limiting their effectiveness and inconspicuousness. In this work, we introduce Layer Embedding Activation Mapping (LEAM), a novel technique that identifies which facial areas contribute most to recognition at an individual level. Unlike adversarial attack methods that aim to fool recognition systems, LEAM is an explainability technique designed to understand how these systems work, providing insights that could inform future privacy protection research. We integrate LEAM with a face parser to analyze data from 1000 individuals across 9 pre-trained facial recognition models. Our analysis reveals that while different layers within facial recognition models vary significantly in their focus areas, these models generally prioritize similar facial regions across architectures when considering their overall activation patterns, which show significantly higher similarity between images of the same individual (Bhattacharyya Coefficient: 0.32-0.57) vs. different individuals (0.04-0.13), validating the existence of person-specific recognition patterns. Our results show that facial recognition models prioritize the central region of face images (with nose areas accounting for 18.9-29.7% of critical recognition regions), while still distributing attention across multiple facial fragments. Proper selection of relevant facial areas was confirmed using validation occlusions, based on just 1% of the most relevant, LEAM-identified, image pixels, which proved to be transferable across different models. Our findings establish the foundation for future individually tailored privacy protection systems centered around LEAM's choice of areas to be perturbed.