mlsanityData to 7 Oct 2026

Benchmarking Joint Face Spoofing and Forgery Detection with Visual and Physiological Cues

arXiv 2208.05401 ↗First posted 10 Aug 2022cs.CVcited by 24

Not on the map. Why?

Abstract (arXiv)

Face anti-spoofing (FAS) and face forgery detection play vital roles in securing face biometric systems from presentation attacks (PAs) and vicious digital manipulation (e.g., deepfakes). Despite promising performance upon large-scale data and powerful deep models, the generalization problem of existing approaches is still an open issue. Most of recent approaches focus on 1) unimodal visual appearance or physiological (i.e., remote photoplethysmography (rPPG)) cues; and 2) separated feature representation for FAS or face forgery detection. On one side, unimodal appearance and rPPG features are respectively vulnerable to high-fidelity face 3D mask and video replay attacks, inspiring us to design reliable multi-modal fusion mechanisms for generalized face attack detection. On the other side, there are rich common features across FAS and face forgery detection tasks (e.g., periodic rPPG rhythms and vanilla appearance for bonafides), providing solid evidence to design a joint FAS and face forgery detection system in a multi-task learning fashion. In this paper, we establish the first joint face spoofing and forgery detection benchmark using both visual appearance and physiological rPPG cues. To enhance the rPPG periodicity discrimination, we design a two-branch physiological network using both facial spatio-temporal rPPG signal map and its continuous wavelet transformed counterpart as inputs. To mitigate the modality bias and improve the fusion efficacy, we conduct a weighted batch and layer normalization for both appearance and rPPG features before multi-modal fusion. We find that the generalization capacities of both unimodal (appearance or rPPG) and multi-modal (appearance+rPPG) models can be obviously improved via joint training on these two tasks. We hope this new benchmark will facilitate the future research of both FAS and deepfake detection communities.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 67 of its 99 references that are in the map. First posted on arXiv 10 Aug 2022.

Builds on

67 of 99 references.

References · 67

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2016Layer Normalizationcited by 5,579
  5. 2017Squeeze-and-Excitation Networkscited by 4,763
  6. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
  7. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  8. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
  9. 2016Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networkscited by 895
  10. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  11. 2018Taskonomy: Disentangling Task Transfer Learningcited by 631
  12. 2017Generative Adversarial Networks: An Overviewcited by 588
38 more
  1. 2019Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensicscited by 573
  2. 2020Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Cluescited by 518
  3. 2019Face X-ray for More General Face Forgery Detectioncited by 469
  4. 2018MesoNet: a Compact Facial Video Forgery Detection Networkcited by 409
  5. 2021Multi-attentional Deepfake Detectioncited by 339
  6. 2021Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domaincited by 241
  7. 2020DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detectioncited by 231
  8. 2019The Deepfake Detection Challenge (DFDC) Preview Datasetcited by 208
  9. 2016UberNet: Training a `Universal' Convolutional Neural Network for Low-, Mid-, and High-Level Vision using Diverse Datasets and Limited Memorycited by 200
  10. 2018Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videoscited by 180
  11. 2020Searching Central Difference Convolutional Networks for Face Anti-Spoofingcited by 173
  12. 2022Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detectioncited by 172
  13. 2018Learning Deep Models for Face Anti-Spoofing: Binary or Auxiliary Supervisioncited by 167
  14. 2019FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signalscited by 143
  15. 2020Single-Side Domain Generalization for Face Anti-Spoofingcited by 139
  16. 2021Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detectioncited by 133
  17. 2018Two-Stream Neural Networks for Tampered Face Detectioncited by 125
  18. 2021PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformercited by 118
  19. 2020CelebA-Spoof: Large-Scale Face Anti-Spoofing Dataset with Rich Annotationscited by 109
  20. 2020NAS-FAS: Static-Dynamic Central Difference Network Search for Face Anti-Spoofingcited by 104
  21. 2020Cross-domain Face Presentation Attack Detection via Multi-domain Disentangled Representation Learningcited by 93
  22. 2019Remote Heart Rate Measurement from Highly Compressed Facial Videos: an End-to-end Deep Learning Solution with Video Enhancementcited by 86
  23. 2019Regularized Fine-grained Meta Face Anti-spoofingcited by 86
  24. 2015face anti-spoofing based on color texture analysiscited by 82
  25. 2019Biometric Face Presentation Attack Detection with Multi-Channel Convolutional Neural Networkcited by 78
  26. 2020Video-based Remote Physiological Measurement via Cross-verified Feature Disentanglingcited by 78
  27. 2019Deep Pixel-wise Binary Supervision for Face Presentation Attack Detectioncited by 76
  28. 2019Deep Tree Learning for Zero-shot Face Anti-Spoofingcited by 73
  29. 2022Domain Generalization via Shuffled Style Assembly for Face Anti-Spoofingcited by 73
  30. 2020Deep Spatial Gradient and Temporal Depth Learning for Face Anti-spoofingcited by 70
  31. 2020DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythmscited by 65
  32. 2020Face Anti-Spoofing Via Disentangled Representation Learningcited by 65
  33. 2021Deep Learning for Face Anti-Spoofing: A Surveycited by 64
  34. 2020Face Anti-Spoofing with Human Material Perceptioncited by 62
  35. 2020On Disentangling Spoof Trace for Generic Face Anti-Spoofingcited by 62
  36. 2018A Dataset and Benchmark for Large-scale Multi-modal Face Anti-spoofingcited by 51
  37. 2021Dual-Cross Central Difference Network for Face Anti-Spoofingcited by 47
  38. 2020Revisiting Pixel-Wise Supervision for Face Anti-Spoofingcited by 45

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 10; 2025: 8; 2026: 1.

24 papers.

Sort:

By area:

  1. 2023Beyond the Prior Forgery Knowledge: Mining Critical Clues for General Face Forgery Detectioncited by 50
  2. 2024SHIELD : An Evaluation Benchmark for Face Spoofing and Forgery Detection with Multimodal Large Language Modelscited by 39
  3. 2024MoE-FFD: Mixture of Experts for Generalized and Parameter-Efficient Face Forgery Detectioncited by 27
  4. 2024FSFM: A Generalizable Face Security Foundation Model via Self-Supervised Facial Representation Learningcited by 21
  5. 2022Digital and Physical Face Attacks: Reviewing and One Step Furthercited by 17
  6. 2024Unified Physical-Digital Face Attack Detectioncited by 17
  7. 2024GM-DF: Generalized Multi-Scenario Deepfake Detectioncited by 16
  8. 2024Towards Data-Centric Face Anti-Spoofing: Improving Cross-domain Generalization via Physics-based Data Synthesiscited by 16
  9. 2023Rehearsal-Free Domain Continual Face Anti-Spoofing: Generalize More and Forget Lesscited by 12
  10. 2024La-SoftMoE CLIP for Unified Physical-Digital Face Attack Detectioncited by 8
  11. 2025FaceShield: Explainable Face Anti-Spoofing with Multimodal Large Language Modelscited by 6
  12. 2023M3FAS: An Accurate and Robust MultiModal Mobile Face Anti-Spoofing Systemcited by 5
12 more
  1. 2025SLIP: Spoof-Aware One-Class Face Anti-Spoofing with Language Image Pretrainingcited by 5
  2. 2025Mixture-of-Attack-Experts with Class Regularization for Unified Physical-Digital Face Attack Detectioncited by 5
  3. 2025Benchmarking Unified Face Attack Detection via Hierarchical Prompt Tuningcited by 5
  4. 2024Resolve Domain Conflicts for Generalizable Remote Physiological Measurementcited by 4
  5. 2024Unified Physical-Digital Attack Detection Challengecited by 3
  6. 2025FA^{3}-CLIP: Frequency-Aware Cues Fusion and Attack-Agnostic Prompt Learning for Unified Face Attack Detectioncited by 3
  7. 2025SUEDE:Shared Unified Experts for Physical-Digital Face Attack Detection Enhancementcited by 2
  8. 2022Face Presentation Attack Detectioncited by 0
  9. 2024FaceCat: Enhancing Face Recognition Security with a Unified Diffusion Modelcited by 0
  10. 2025Paired-Sampling Contrastive Framework for Joint Physical-Digital Face Attack Detectioncited by 0
  11. 2025Spoofing-aware Prompt Learning for Unified Physical-Digital Facial Attack Detectioncited by 0
  12. 2026UniShield: Unified Face Attack Detection via KG-Informed Multimodal Reasoningcited by 0