# A Responsible Face Recognition Approach for Small and Mid-Scale Systems Through Personalized Neural Networks

arXiv: https://arxiv.org/abs/2505.19920 · HTML: https://mlsanity.com/paper/2505.19920/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 61 references that are in the map.
- First posted on arXiv 26 May 2025.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [MagFace: A Universal Representation for Face Recognition and Quality Assessment](https://mlsanity.com/paper/2103.06627/) (2021)
- [Deep Face Recognition: A Survey](https://mlsanity.com/paper/1804.06655/) (2018)
- [FairGAN: Fairness-aware Generative Adversarial Networks](https://mlsanity.com/paper/1805.11202/) (2018)
- [Demographic Bias in Biometrics: A Survey on an Emerging Challenge](https://mlsanity.com/paper/2003.02488/) (2020)
- [FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age](https://mlsanity.com/paper/1908.04913/) (2019)
- [A Comprehensive Study on Face Recognition Biases Beyond Demographics](https://mlsanity.com/paper/2103.01592/) (2021)
- [Face Transformer for Recognition](https://mlsanity.com/paper/2103.14803/) (2021)
- [Fairness in Biometrics: a figure of merit to assess biometric verification systems](https://mlsanity.com/paper/2011.02395/) (2020)
- [Characterizing the Variability in Face Recognition Accuracy Relative to Race](https://mlsanity.com/paper/1904.07325/) (2019)
- [Explainable Face Recognition](https://mlsanity.com/paper/2008.00916/) (2020)
- [Gendered Differences in Face Recognition Accuracy Explained by Hairstyles, Makeup, and Facial Morphology](https://mlsanity.com/paper/2112.14656/) (2021)
- [Post-Comparison Mitigation of Demographic Bias in Face Recognition Using Fair Score Normalization](https://mlsanity.com/paper/2002.03592/) (2020)
- [Training Unbiased Diffusion Models From Biased Dataset](https://mlsanity.com/paper/2403.01189/) (2024)
- [Beyond Identity: What Information Is Stored in Biometric Face Templates?](https://mlsanity.com/paper/2009.09918/) (2020)
- [Evaluating Proposed Fairness Models for Face Recognition Algorithms](https://mlsanity.com/paper/2203.05051/) (2022)
- [Part-based Face Recognition with Vision Transformers](https://mlsanity.com/paper/2212.00057/) (2022)
- [Pixel-Level Face Image Quality Assessment for Explainable Face Recognition](https://mlsanity.com/paper/2110.11001/) (2021)
- [Template Adaptation for Face Verification and Identification](https://mlsanity.com/paper/1603.03958/) (2016)
- [Efficient Explainable Face Verification based on Similarity Score Argument Backpropagation](https://mlsanity.com/paper/2304.13409/) (2023)
- [An Attack on Facial Soft-biometric Privacy Enhancement](https://mlsanity.com/paper/2111.12405/) (2021)
- [On the Reliability of Biometric Datasets: How Much Test Data Ensures Reliability?](https://mlsanity.com/paper/2501.06504/) (2025)
- [Unsupervised Enhancement of Soft-biometric Privacy with Negative Face Recognition](https://mlsanity.com/paper/2002.09181/) (2020)
- [LabellessFace: Fair Metric Learning for Face Recognition without Attribute Labels](https://mlsanity.com/paper/2409.09274/) (2024)
- [On the Illusion of Gender Bias in Face Recognition: Explaining the Fairness Issue Through Non-demographic Attributes](https://mlsanity.com/paper/2501.12020/) (2025)

## Abstract (arXiv)

Traditional face recognition systems rely on extracting fixed face representations, known as templates, to store and verify identities. These representations are typically generated by neural networks that often lack explainability and raise concerns regarding fairness and privacy. In this work, we propose a novel model-template (MOTE) approach that replaces vector-based face templates with small personalized neural networks. This design enables more responsible face recognition for small and medium-scale systems. During enrollment, MOTE creates a dedicated binary classifier for each identity, trained to determine whether an input face matches the enrolled identity. Each classifier is trained using only a single reference sample, along with synthetically balanced samples to allow adjusting fairness at the level of a single individual during enrollment. Extensive experiments across multiple datasets and recognition systems demonstrate substantial improvements in fairness and particularly in privacy. Although the method increases inference time and storage requirements, it presents a strong solution for small- and mid-scale applications where fairness and privacy are critical.