mlsanityData to 14 Oct 2026

FM-ViT: Flexible Modal Vision Transformers for Face Anti-Spoofing

arXiv 2305.03277 ↗First posted 5 May 2023cs.CVcited by 22

Not on the map. Why?

Abstract (arXiv)

The availability of handy multi-modal (i.e., RGB-D) sensors has brought about a surge of face anti-spoofing research. However, the current multi-modal face presentation attack detection (PAD) has two defects: (1) The framework based on multi-modal fusion requires providing modalities consistent with the training input, which seriously limits the deployment scenario. (2) The performance of ConvNet-based model on high fidelity datasets is increasingly limited. In this work, we present a pure transformer-based framework, dubbed the Flexible Modal Vision Transformer (FM-ViT), for face anti-spoofing to flexibly target any single-modal (i.e., RGB) attack scenarios with the help of available multi-modal data. Specifically, FM-ViT retains a specific branch for each modality to capture different modal information and introduces the Cross-Modal Transformer Block (CMTB), which consists of two cascaded attentions named Multi-headed Mutual-Attention (MMA) and Fusion-Attention (MFA) to guide each modal branch to mine potential features from informative patch tokens, and to learn modality-agnostic liveness features by enriching the modal information of own CLS token, respectively. Experiments demonstrate that the single model trained based on FM-ViT can not only flexibly evaluate different modal samples, but also outperforms existing single-modal frameworks by a large margin, and approaches the multi-modal frameworks introduced with smaller FLOPs and model parameters.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 47 of its 72 references that are in the map. First posted on arXiv 5 May 2023.

Builds on

47 of 72 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2020End-to-End Object Detection with Transformerscited by 7,414
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  8. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  9. 2020Transformer Interpretability Beyond Attention Visualizationcited by 627
  10. 2018Video Action Transformer Networkcited by 316
  11. 2021UniT: Multimodal Multitask Learning with a Unified Transformercited by 195
  12. 2020Searching Central Difference Convolutional Networks for Face Anti-Spoofingcited by 173
35 more
  1. 2018Learning Deep Models for Face Anti-Spoofing: Binary or Auxiliary Supervisioncited by 167
  2. 2020CelebA-Spoof: Large-Scale Face Anti-Spoofing Dataset with Rich Annotationscited by 109
  3. 2020NAS-FAS: Static-Dynamic Central Difference Network Search for Face Anti-Spoofingcited by 105
  4. 2020Cross-domain Face Presentation Attack Detection via Multi-domain Disentangled Representation Learningcited by 93
  5. 2015face anti-spoofing based on color texture analysiscited by 82
  6. 2014Learn Convolutional Neural Network for Face Anti-Spoofingcited by 80
  7. 2019Biometric Face Presentation Attack Detection with Multi-Channel Convolutional Neural Networkcited by 79
  8. 2019Deep Pixel-wise Binary Supervision for Face Presentation Attack Detectioncited by 76
  9. 2020Deep Spatial Gradient and Temporal Depth Learning for Face Anti-spoofingcited by 70
  10. 2022PatchNet: A Simple Face Anti-Spoofing Framework via Fine-Grained Patch Recognitioncited by 67
  11. 2020CASIA-SURF CeFA: A Benchmark for Multi-modal Cross-ethnicity Face Anti-spoofingcited by 66
  12. 2020Face Anti-Spoofing Via Disentangled Representation Learningcited by 65
  13. 2021KVT: k-NN Attention for Boosting Vision Transformerscited by 65
  14. 2020Face Anti-Spoofing with Human Material Perceptioncited by 62
  15. 2020On Disentangling Spoof Trace for Generic Face Anti-Spoofingcited by 62
  16. 2019CASIA-SURF: A Large-scale Multi-modal Benchmark for Face Anti-spoofingcited by 61
  17. 2018Face De-Spoofing: Anti-Spoofing via Noise Modelingcited by 60
  18. 2018A Dataset and Benchmark for Large-scale Multi-modal Face Anti-spoofingcited by 51
  19. 2020On the Effectiveness of Vision Transformers for Zero-shot Face Anti-Spoofingcited by 44
  20. 2021Contrastive Context-Aware Learning for 3D High-Fidelity Mask Face Presentation Attack Detectioncited by 43
  21. 2021Cross Modal Focal Loss for RGBD Face Anti-Spoofingcited by 41
  22. 2021TransRPPG: Remote Photoplethysmography Transformer for 3D Mask Face Presentation Attack Detectioncited by 41
  23. 2022Adaptive Transformers for Robust Few-shot Cross-domain Face Anti-spoofingcited by 38
  24. 2020Multi-Modal Face Anti-Spoofing Based on Central Difference Networkscited by 36
  25. 2020Learning One Class Representations for Face Presentation Attack Detection using Multi-channel Convolutional Neural Networkscited by 36
  26. 2021Meta-Teacher For Face Anti-Spoofingcited by 33
  27. 2020Deep Models and Shortwave Infrared Information to Detect Face Presentation Attackscited by 29
  28. 2021Learning Meta Pattern for Face Anti-Spoofingcited by 29
  29. 2020Cross-ethnicity Face Anti-spoofing Recognition Challenge: A Reviewcited by 25
  30. 2022Flexible-Modal Face Anti-Spoofing: A Benchmarkcited by 21
  31. 20213D High-Fidelity Mask Face Presentation Attack Detection Challengecited by 19
  32. 2023MA-ViT: Modality-Agnostic Vision Transformers for Face Anti-Spoofingcited by 18
  33. 2020Noise Modeling, Synthesis and Classification for Generic Object Anti-Spoofingcited by 14
  34. 2023Surveillance Face Anti-spoofingcited by 12
  35. 2020The High-Quality Wide Multi-Channel Attack (HQ-WMCA) databasecited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 8; 2025: 7; 2026: 2.

22 papers.

Sort:

By area:

  1. 2023Forgery-aware Adaptive Transformer for Generalizable Synthetic Image Detectioncited by 182
  2. 2024CFPL-FAS: Class Free Prompt Learning for Generalizable Face Anti-spoofingcited by 39
  3. 2024Test-Time Domain Generalization for Face Anti-Spoofingcited by 20
  4. 2024Unified Physical-Digital Face Attack Detectioncited by 17
  5. 2023Unified Frequency-Assisted Transformer Framework for Detecting and Grounding Multi-Modal Manipulationcited by 14
  6. 2023Visual Prompt Flexible-Modal Face Anti-Spoofingcited by 9
  7. 2024Joint Physical-Digital Facial Attack Detection Via Simulating Spoofing Cluescited by 8
  8. 2024La-SoftMoE CLIP for Unified Physical-Digital Face Attack Detectioncited by 8
  9. 2025Domain Generalization for Face Anti-spoofing via Content-aware Composite Prompt Engineeringcited by 7
  10. 2023Fine-Grained Annotation for Face Anti-Spoofingcited by 4
  11. 2025DADM: Dual Alignment of Domain and Modality for Face Anti-spoofingcited by 4
  12. 2024Unified Physical-Digital Attack Detection Challengecited by 3
10 more
  1. 2025FA^{3}-CLIP: Frequency-Aware Cues Fusion and Attack-Agnostic Prompt Learning for Unified Face Attack Detectioncited by 3
  2. 2025SUEDE:Shared Unified Experts for Physical-Digital Face Attack Detection Enhancementcited by 2
  3. 2024Liveness Detection in Computer Vision: Transformer-based Self-Supervised Learning for Face Anti-Spoofingcited by 1
  4. 2026PriorNet: Prior-Guided Engagement Estimation from Face Videocited by 1
  5. 2023CLIPC8: Face liveness detection algorithm based on image-text pairs and contrastive learningcited by 0
  6. 2024Spoof Trace Discovery for Deep Learning Based Explainable Face Anti-Spoofingcited by 0
  7. 2025Multi-Modal Face Anti-Spoofing via Cross-Modal Feature Transitionscited by 0
  8. 2025Robust Multi-Modal Face Anti-Spoofing with Domain Adaptation: Tackling Missing Modalities, Noisy Pseudo-Labels, and Model Degradationcited by 0
  9. 2025Spoofing-aware Prompt Learning for Unified Physical-Digital Facial Attack Detectioncited by 0
  10. 2026Architectural Bias in Face Presentation Attack Detection: A Comparative Study of Vision Transformers and Convolutional Neural Networkscited by 0