# Visual Psychophysics for Making Face Recognition Algorithms More Explainable

arXiv: https://arxiv.org/abs/1803.07140 · HTML: https://mlsanity.com/paper/1803.07140/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Mar 2018.

## Built on by (most cited first, 19 of 19)

- [CYBORG: Blending Human Saliency Into the Loss Improves Deep Learning](https://mlsanity.com/paper/2112.00686/) (2021)
- [Human-Aided Saliency Maps Improve Generalization of Deep Learning](https://mlsanity.com/paper/2105.03492/) (2021)
- [Unravelling the Effect of Image Distortions for Biased Prediction of Pre-trained Face Recognition Models](https://mlsanity.com/paper/2108.06581/) (2021)
- [Fair SA: Sensitivity Analysis for Fairness in Face Recognition](https://mlsanity.com/paper/2202.03586/) (2022)
- [Can Transformer Language Models Predict Psychometric Properties?](https://mlsanity.com/paper/2106.06849/) (2021)
- [Motif Mining: Finding and Summarizing Remixed Image Content](https://mlsanity.com/paper/2203.08327/) (2022)
- [Measuring Human Perception to Improve Open Set Recognition](https://mlsanity.com/paper/2209.03519/) (2022)
- [Using Human Perception to Regularize Transfer Learning](https://mlsanity.com/paper/2211.07885/) (2022)
- [Interpreting Face Inference Models using Hierarchical Network Dissection](https://mlsanity.com/paper/2108.10360/) (2021)
- [Guiding Machine Perception with Psychophysics](https://mlsanity.com/paper/2207.02241/) (2022)
- [A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challenges](https://mlsanity.com/paper/2211.06665/) (2022)
- [Psychophysical-Score: A Behavioral Measure for Assessing the Biological Plausibility of Visual Recognition Models](https://mlsanity.com/paper/2210.08632/) (2022)
- [Improving Model's Focus Improves Performance of Deep Learning-Based Synthetic Face Detectors](https://mlsanity.com/paper/2303.00818/) (2023)
- [Explaining Deep Face Algorithms through Visualization: A Survey](https://mlsanity.com/paper/2309.14715/) (2023)
- [Explainable offline automatic signature verifier to support forensic handwriting examiners](https://mlsanity.com/paper/2405.12695/) (2024)
- [Training Better Deep Learning Models Using Human Saliency](https://mlsanity.com/paper/2410.16190/) (2024)
- [Psych-Occlusion: Using Visual Psychophysics for Aerial Detection of Occluded Persons during Search and Rescue](https://mlsanity.com/paper/2412.05553/) (2024)
- [Generalist Multimodal LLMs Gain Biometric Expertise via Human Salience](https://mlsanity.com/paper/2603.17173/) (2026)
- [VISER: Visually-Informed System for Enhanced Robustness in Iris Presentation Attack Detection](https://mlsanity.com/paper/2603.17859/) (2026)

## Abstract (arXiv)

Scientific fields that are interested in faces have developed their own sets of concepts and procedures for understanding how a target model system (be it a person or algorithm) perceives a face under varying conditions. In computer vision, this has largely been in the form of dataset evaluation for recognition tasks where summary statistics are used to measure progress. While aggregate performance has continued to improve, understanding individual causes of failure has been difficult, as it is not always clear why a particular face fails to be recognized, or why an impostor is recognized by an algorithm. Importantly, other fields studying vision have addressed this via the use of visual psychophysics: the controlled manipulation of stimuli and careful study of the responses they evoke in a model system. In this paper, we suggest that visual psychophysics is a viable methodology for making face recognition algorithms more explainable. A comprehensive set of procedures is developed for assessing face recognition algorithm behavior, which is then deployed over state-of-the-art convolutional neural networks and more basic, yet still widely used, shallow and handcrafted feature-based approaches.