# Visual Prompt Flexible-Modal Face Anti-Spoofing

arXiv: https://arxiv.org/abs/2307.13958 · HTML: https://mlsanity.com/paper/2307.13958/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 74 references that are in the map.
- First posted on arXiv 26 Jul 2023.

## Builds on (most cited first, 30 of 57)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition](https://mlsanity.com/paper/2205.13535/) (2022)
- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [TransReID: Transformer-based Object Re-Identification](https://mlsanity.com/paper/2102.04378/) (2021)
- [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021)
- [Attention Bottlenecks for Multimodal Fusion](https://mlsanity.com/paper/2107.00135/) (2021)
- [VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks](https://mlsanity.com/paper/2112.06825/) (2021)
- [Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning](https://mlsanity.com/paper/2210.08823/) (2022)
- [MultiMAE: Multi-modal Multi-task Masked Autoencoders](https://mlsanity.com/paper/2204.01678/) (2022)
- [General Facial Representation Learning in a Visual-Linguistic Manner](https://mlsanity.com/paper/2112.03109/) (2021)
- [Searching Central Difference Convolutional Networks for Face Anti-Spoofing](https://mlsanity.com/paper/2003.04092/) (2020)
- [Learning Deep Models for Face Anti-Spoofing: Binary or Auxiliary Supervision](https://mlsanity.com/paper/1803.11097/) (2018)
- [Visual Prompt Multi-Modal Tracking](https://mlsanity.com/paper/2303.10826/) (2023)
- [Single-Side Domain Generalization for Face Anti-Spoofing](https://mlsanity.com/paper/2004.14043/) (2020)
- [Multimodal Prompting with Missing Modalities for Visual Recognition](https://mlsanity.com/paper/2303.03369/) (2023)
- [face anti-spoofing based on color texture analysis](https://mlsanity.com/paper/1511.06316/) (2015)
- [Biometric Face Presentation Attack Detection with Multi-Channel Convolutional Neural Network](https://mlsanity.com/paper/1909.08848/) (2019)

## Built on by (most cited first, 9 of 9)

- [Suppress and Rebalance: Towards Generalized Multi-Modal Face Anti-Spoofing](https://mlsanity.com/paper/2402.19298/) (2024)
- [La-SoftMoE CLIP for Unified Physical-Digital Face Attack Detection](https://mlsanity.com/paper/2408.12793/) (2024)
- [Domain Generalization for Face Anti-spoofing via Content-aware Composite Prompt Engineering](https://mlsanity.com/paper/2504.04470/) (2025)
- [Benchmarking Unified Face Attack Detection via Hierarchical Prompt Tuning](https://mlsanity.com/paper/2505.13327/) (2025)
- [DADM: Dual Alignment of Domain and Modality for Face Anti-spoofing](https://mlsanity.com/paper/2503.00429/) (2025)
- [Learning Knowledge-based Prompts for Robust 3D Mask Presentation Attack Detection](https://mlsanity.com/paper/2505.03610/) (2025)
- [BIG-MoE: Bypass Isolated Gating MoE for Generalized Multimodal Face Anti-Spoofing](https://mlsanity.com/paper/2412.18065/) (2024)
- [AI-driven Remote Facial Skin Hydration and TEWL Assessment from Selfie Images: A Systematic Solution](https://mlsanity.com/paper/2509.06282/) (2025)
- [Robust Multi-Modal Face Anti-Spoofing with Domain Adaptation: Tackling Missing Modalities, Noisy Pseudo-Labels, and Model Degradation](https://mlsanity.com/paper/2509.23475/) (2025)

## Abstract (arXiv)

Recently, vision transformer based multimodal learning methods have been proposed to improve the robustness of face anti-spoofing (FAS) systems. However, multimodal face data collected from the real world is often imperfect due to missing modalities from various imaging sensors. Recently, flexible-modal FAS~citeyu2023flexible has attracted more attention, which aims to develop a unified multimodal FAS model using complete multimodal face data but is insensitive to test-time missing modalities. In this paper, we tackle one main challenge in flexible-modal FAS, i.e., when missing modality occurs either during training or testing in real-world situations. Inspired by the recent success of the prompt learning in language models, we propose Visual Prompt flexible-modal FAS (VP-FAS), which learns the modal-relevant prompts to adapt the frozen pre-trained foundation model to downstream flexible-modal FAS task. Specifically, both vanilla visual prompts and residual contextual prompts are plugged into multimodal transformers to handle general missing-modality cases, while only requiring less than 4% learnable parameters compared to training the entire model. Furthermore, missing-modality regularization is proposed to force models to learn consistent multimodal feature embeddings when missing partial modalities. Extensive experiments conducted on two multimodal FAS benchmark datasets demonstrate the effectiveness of our VP-FAS framework that improves the performance under various missing-modality cases while alleviating the requirement of heavy model re-training.