# Explaining Face Presentation Attack Detection Using Natural Language

arXiv: https://arxiv.org/abs/2111.04862 · HTML: https://mlsanity.com/paper/2111.04862/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 56 references that are in the map.
- First posted on arXiv 8 Nov 2021.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioning](https://mlsanity.com/paper/1611.05594/) (2016)
- [Searching Central Difference Convolutional Networks for Face Anti-Spoofing](https://mlsanity.com/paper/2003.04092/) (2020)
- [Learning Deep Models for Face Anti-Spoofing: Binary or Auxiliary Supervision](https://mlsanity.com/paper/1803.11097/) (2018)
- [Grad-CAM: Why did you say that?](https://mlsanity.com/paper/1611.07450/) (2016)
- [Generating Visual Explanations](https://mlsanity.com/paper/1603.08507/) (2016)
- [Learning a Deep Embedding Model for Zero-Shot Learning](https://mlsanity.com/paper/1611.05088/) (2016)
- [face anti-spoofing based on color texture analysis](https://mlsanity.com/paper/1511.06316/) (2015)
- [Learn Convolutional Neural Network for Face Anti-Spoofing](https://mlsanity.com/paper/1408.5601/) (2014)
- [Explainable Artificial Intelligence: a Systematic Review](https://mlsanity.com/paper/2006.00093/) (2020)
- [Deep Spatial Gradient and Temporal Depth Learning for Face Anti-spoofing](https://mlsanity.com/paper/2003.08061/) (2020)
- [Face De-Spoofing: Anti-Spoofing via Noise Modeling](https://mlsanity.com/paper/1807.09968/) (2018)
- [Learning Robust Visual-Semantic Embeddings](https://mlsanity.com/paper/1703.05908/) (2017)
- [Learning Meta Model for Zero- and Few-shot Face Anti-spoofing](https://mlsanity.com/paper/1904.12490/) (2019)
- [On the Effectiveness of Vision Transformers for Zero-shot Face Anti-Spoofing](https://mlsanity.com/paper/2011.08019/) (2020)
- [Explainable Face Recognition](https://mlsanity.com/paper/2008.00916/) (2020)
- [Detection and Continual Learning of Novel Face Presentation Attacks](https://mlsanity.com/paper/2108.12081/) (2021)
- [Image Captioning through Image Transformer](https://mlsanity.com/paper/2004.14231/) (2020)

## Built on by (most cited first, 2 of 2)

- [Are Explainability Tools Gender Biased? A Case Study on Face Presentation Attack Detection](https://mlsanity.com/paper/2304.13419/) (2023)
- [Fine-Grained Annotation for Face Anti-Spoofing](https://mlsanity.com/paper/2310.08142/) (2023)

## Abstract (arXiv)

A large number of deep neural network based techniques have been developed to address the challenging problem of face presentation attack detection (PAD). Whereas such techniques' focus has been on improving PAD performance in terms of classification accuracy and robustness against unseen attacks and environmental conditions, there exists little attention on the explainability of PAD predictions. In this paper, we tackle the problem of explaining PAD predictions through natural language. Our approach passes feature representations of a deep layer of the PAD model to a language model to generate text describing the reasoning behind the PAD prediction. Due to the limited amount of annotated data in our study, we apply a light-weight LSTM network as our natural language generation model. We investigate how the quality of the generated explanations is affected by different loss functions, including the commonly used word-wise cross entropy loss, a sentence discriminative loss, and a sentence semantic loss. We perform our experiments using face images from a dataset consisting of 1,105 bona-fide and 924 presentation attack samples. Our quantitative and qualitative results show the effectiveness of our model for generating proper PAD explanations through text as well as the power of the sentence-wise losses. To the best of our knowledge, this is the first introduction of a joint biometrics-NLP task. Our dataset can be obtained through our GitHub page.