# MA-ViT: Modality-Agnostic Vision Transformers for Face Anti-Spoofing

arXiv: https://arxiv.org/abs/2304.07549 · HTML: https://mlsanity.com/paper/2304.07549/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 25 references that are in the map.
- First posted on arXiv 15 Apr 2023.

## Builds on (most cited first, 19 of 19)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Searching Central Difference Convolutional Networks for Face Anti-Spoofing](https://mlsanity.com/paper/2003.04092/) (2020)
- [Learning Deep Models for Face Anti-Spoofing: Binary or Auxiliary Supervision](https://mlsanity.com/paper/1803.11097/) (2018)
- [CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2109.06165/) (2021)
- [Biometric Face Presentation Attack Detection with Multi-Channel Convolutional Neural Network](https://mlsanity.com/paper/1909.08848/) (2019)
- [Deep Pixel-wise Binary Supervision for Face Presentation Attack Detection](https://mlsanity.com/paper/1907.04047/) (2019)
- [CASIA-SURF CeFA: A Benchmark for Multi-modal Cross-ethnicity Face Anti-spoofing](https://mlsanity.com/paper/2003.05136/) (2020)
- [Face Anti-Spoofing Via Disentangled Representation Learning](https://mlsanity.com/paper/2008.08250/) (2020)
- [KVT: k-NN Attention for Boosting Vision Transformers](https://mlsanity.com/paper/2106.00515/) (2021)
- [On Disentangling Spoof Trace for Generic Face Anti-Spoofing](https://mlsanity.com/paper/2007.09273/) (2020)
- [CASIA-SURF: A Large-scale Multi-modal Benchmark for Face Anti-spoofing](https://mlsanity.com/paper/1908.10654/) (2019)
- [A Dataset and Benchmark for Large-scale Multi-modal Face Anti-spoofing](https://mlsanity.com/paper/1812.00408/) (2018)
- [Contrastive Context-Aware Learning for 3D High-Fidelity Mask Face Presentation Attack Detection](https://mlsanity.com/paper/2104.06148/) (2021)
- [Cross Modal Focal Loss for RGBD Face Anti-Spoofing](https://mlsanity.com/paper/2103.00948/) (2021)
- [Learning One Class Representations for Face Presentation Attack Detection using Multi-channel Convolutional Neural Networks](https://mlsanity.com/paper/2007.11457/) (2020)
- [Cross-ethnicity Face Anti-spoofing Recognition Challenge: A Review](https://mlsanity.com/paper/2004.10998/) (2020)
- [3D High-Fidelity Mask Face Presentation Attack Detection Challenge](https://mlsanity.com/paper/2108.06968/) (2021)

## Built on by (most cited first, 18 of 18)

- [Forgery-aware Adaptive Transformer for Generalizable Synthetic Image Detection](https://mlsanity.com/paper/2312.16649/) (2023)
- [SHIELD : An Evaluation Benchmark for Face Spoofing and Forgery Detection with Multimodal Large Language Models](https://mlsanity.com/paper/2402.04178/) (2024)
- [CFPL-FAS: Class Free Prompt Learning for Generalizable Face Anti-spoofing](https://mlsanity.com/paper/2403.14333/) (2024)
- [FM-ViT: Flexible Modal Vision Transformers for Face Anti-Spoofing](https://mlsanity.com/paper/2305.03277/) (2023)
- [Test-Time Domain Generalization for Face Anti-Spoofing](https://mlsanity.com/paper/2403.19334/) (2024)
- [Unified Physical-Digital Face Attack Detection](https://mlsanity.com/paper/2401.17699/) (2024)
- [Unified Frequency-Assisted Transformer Framework for Detecting and Grounding Multi-Modal Manipulation](https://mlsanity.com/paper/2309.09667/) (2023)
- [Visual Prompt Flexible-Modal Face Anti-Spoofing](https://mlsanity.com/paper/2307.13958/) (2023)
- [Joint Physical-Digital Facial Attack Detection Via Simulating Spoofing Clues](https://mlsanity.com/paper/2404.08450/) (2024)
- [La-SoftMoE CLIP for Unified Physical-Digital Face Attack Detection](https://mlsanity.com/paper/2408.12793/) (2024)
- [Domain Generalization for Face Anti-spoofing via Content-aware Composite Prompt Engineering](https://mlsanity.com/paper/2504.04470/) (2025)
- [DADM: Dual Alignment of Domain and Modality for Face Anti-spoofing](https://mlsanity.com/paper/2503.00429/) (2025)
- [Unified Physical-Digital Attack Detection Challenge](https://mlsanity.com/paper/2404.06211/) (2024)
- [FA^{3}-CLIP: Frequency-Aware Cues Fusion and Attack-Agnostic Prompt Learning for Unified Face Attack Detection](https://mlsanity.com/paper/2504.00454/) (2025)
- [SUEDE:Shared Unified Experts for Physical-Digital Face Attack Detection Enhancement](https://mlsanity.com/paper/2504.04818/) (2025)
- [Hyperbolic Face Anti-Spoofing](https://mlsanity.com/paper/2308.09107/) (2023)
- [BIG-MoE: Bypass Isolated Gating MoE for Generalized Multimodal Face Anti-Spoofing](https://mlsanity.com/paper/2412.18065/) (2024)
- [CLIPC8: Face liveness detection algorithm based on image-text pairs and contrastive learning](https://mlsanity.com/paper/2311.17583/) (2023)

## Abstract (arXiv)

The existing multi-modal face anti-spoofing (FAS) frameworks are designed based on two strategies: halfway and late fusion. However, the former requires test modalities consistent with the training input, which seriously limits its deployment scenarios. And the latter is built on multiple branches to process different modalities independently, which limits their use in applications with low memory or fast execution requirements. In this work, we present a single branch based Transformer framework, namely Modality-Agnostic Vision Transformer (MA-ViT), which aims to improve the performance of arbitrary modal attacks with the help of multi-modal data. Specifically, MA-ViT adopts the early fusion to aggregate all the available training modalities data and enables flexible testing of any given modal samples. Further, we develop the Modality-Agnostic Transformer Block (MATB) in MA-ViT, which consists of two stacked attentions named Modal-Disentangle Attention (MDA) and Cross-Modal Attention (CMA), to eliminate modality-related information for each modal sequences and supplement modality-agnostic liveness features from another modal sequences, respectively. Experiments demonstrate that the single model trained based on MA-ViT can not only flexibly evaluate different modal samples, but also outperforms existing single-modal frameworks by a large margin, and approaches the multi-modal frameworks introduced with smaller FLOPs and model parameters.