mlsanityData to 13 Oct 2026

M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection

arXiv 2104.09770 ↗First posted 20 Apr 2021cs.CVcited by 114

Not on the map. Why?

Abstract (arXiv)

The widespread dissemination of Deepfakes demands effective approaches that can detect perceptually convincing forged images. In this paper, we aim to capture the subtle manipulation artifacts at different scales using transformer models. In particular, we introduce a Multi-modal Multi-scale TRansformer (M2TR), which operates on patches of different sizes to detect local inconsistencies in images at different spatial levels. M2TR further learns to detect forgery artifacts in the frequency domain to complement RGB information through a carefully designed cross modality fusion block. In addition, to stimulate Deepfake detection research, we introduce a high-quality Deepfake dataset, SR-DF, which consists of 4,000 DeepFake videos generated by state-of-the-art face swapping and facial reenactment methods. We conduct extensive experiments to verify the effectiveness of the proposed method, which outperforms state-of-the-art Deepfake detection methods by clear margins.

Key facts. Cited by 114 papers in the map (data through 13 Oct 2026). Builds on 56 of its 79 references that are in the map. First posted on arXiv 20 Apr 2021.

Builds on

56 of 79 references.

References · 56

  1. 2017Attention Is All You Needcited by 60,106
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,321
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,549
  7. 2020End-to-End Object Detection with Transformerscited by 7,396
  8. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,426
  9. 2020Training data-efficient image transformers & distillation through attentioncited by 4,574
  10. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,417
  11. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,264
  12. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,326
38 more
  1. 2019Searching for MobileNetV3cited by 1,990
  2. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,097
  3. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 967
  4. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 846
  5. 2017VoxCeleb: a large-scale speaker identification datasetcited by 719
  6. 2020First Order Motion Model for Image Animationcited by 702
  7. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 631
  8. 2019Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensicscited by 574
  9. 2020Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Cluescited by 520
  10. 2019Face X-ray for More General Face Forgery Detectioncited by 469
  11. 2020ResNeSt: Split-Attention Networkscited by 445
  12. 2018MesoNet: a Compact Facial Video Forgery Detection Networkcited by 409
  13. 2017Learning Spatio-Temporal Representation with Pseudo-3D Residual Networkscited by 386
  14. 2021Multi-attentional Deepfake Detectioncited by 340
  15. 2019FSGAN: Subject Agnostic Face Swapping and Reenactmentcited by 320
  16. 2018Exposing DeepFake Videos By Detecting Face Warping Artifactscited by 318
  17. 2018Exposing Deep Fakes Using Inconsistent Head Posescited by 296
  18. 2019Deferred Neural Rendering: Image Synthesis using Neural Texturescited by 253
  19. 2019FaceShifter: Towards High Fidelity And Occlusion Aware Face Swappingcited by 221
  20. 2020Global Texture Enhancement for Fake Face Detection in the Wildcited by 220
  21. 2018DeepFakes: a New Threat to Face Recognition? Assessment and Detectioncited by 208
  22. 2021WildDeepfake: A Challenging Real-World Dataset for Deepfake Detectioncited by 208
  23. 2019EdgeConnect: Generative Image Inpainting with Adversarial Edge Learningcited by 163
  24. 2017R-C3D: Region Convolutional 3D Network for Temporal Activity Detectioncited by 161
  25. 2020Two-branch Recurrent Network for Isolating Deepfakes in Videoscited by 161
  26. 2018GANimation: Anatomically-aware Facial Animation from a Single Imagecited by 160
  27. 2019DoveNet: Deep Image Harmonization via Domain Verificationcited by 134
  28. 2019Multi-task Learning For Detecting and Segmenting Manipulated Facial Images and Videoscited by 130
  29. 2018Two-Stream Neural Networks for Tampered Face Detectioncited by 125
  30. 2021ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysiscited by 125
  31. 2017Coherent Online Video Style Transfercited by 96
  32. 2018ReenactGAN: Learning to Reenact Faces via Boundary Transfercited by 85
  33. 2021Deepfake Video Detection Using Convolutional Vision Transformercited by 79
  34. 2019Unmasking DeepFakes with simple Featurescited by 78
  35. 2020Blind Video Temporal Consistency via Deep Video Priorcited by 76
  36. 2019Use of a Capsule Network to Detect Fake Images and Videoscited by 63
  37. 2017Recasting Residual-based Local Descriptors as Convolutional Neural Networks: an Application to Image Forgery Detectioncited by 56
  38. 2016Artistic style transfer for videoscited by 54

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 7; 2022: 19; 2023: 24; 2024: 32; 2025: 21; 2026: 11.

114 papers.

Sort:

By area:

  1. 2023UCF: Uncovering Common Features for Generalizable Deepfake Detectioncited by 177
  2. 2023DeepfakeBench: A Comprehensive Benchmark of Deepfake Detectioncited by 163
  3. 2021DS-TransUNet:Dual Swin Transformer U-Net for Medical Image Segmentationcited by 126
  4. 2022Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalizationcited by 122
  5. 2024Orthogonal Subspace Decomposition for Generalizable AI-Generated Image Detectioncited by 103
  6. 2023TALL: Thumbnail Layout for Deepfake Video Detectioncited by 100
  7. 2023CIFAKE: Image Classification and Explainable Identification of AI-Generated Synthetic Imagescited by 84
  8. 2021Towards Transferable Adversarial Attacks on Vision Transformerscited by 80
  9. 2023AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Datasetcited by 79
  10. 2022ObjectFormer for Image Manipulation Detection and Localizationcited by 78
  11. 2024Deepfake Generation and Detection: A Benchmark and Surveycited by 67
  12. 2021Token Shift Transformer for Video Classificationcited by 61
38 more
  1. 2024Generalizing Deepfake Video Detection with Plug-and-Play: Video-Level Blending and Spatiotemporal Adapter Tuningcited by 58
  2. 2022Prototypical Residual Networks for Anomaly Detection and Localizationcited by 56
  3. 2022MARLIN: Masked Autoencoder for facial video Representation LearnINgcited by 55
  4. 2023Multimodal Automated Fact-Checking: A Surveycited by 52
  5. 2022Audio-Visual Person-of-Interest DeepFake Detectioncited by 43
  6. 2023Glitch in the Matrix: A Large Scale Benchmark for Content Driven Audio-Visual Forgery Detection and Localizationcited by 42
  7. 2022Delving into Sequential Patches for Deepfake Detectioncited by 41
  8. 2023Improving Fairness in Deepfake Detectioncited by 40
  9. 2024MoE-FFD: Mixture of Experts for Generalized and Parameter-Efficient Face Forgery Detectioncited by 27
  10. 2024Circumventing shortcuts in audio-visual deepfake detection datasets with unsupervised learningcited by 23
  11. 2022Self-supervised Transformer for Deepfake Detectioncited by 22
  12. 2024Learning Spatiotemporal Inconsistency via Thumbnail Layout for Face Deepfake Detectioncited by 21
  13. 2024PolyGlotFake: A Novel Multilingual and Multimodal DeepFake Datasetcited by 16
  14. 2024GM-DF: Generalized Multi-Scenario Deepfake Detectioncited by 16
  15. 2021Watch Those Words: Video Falsification Detection Using Word-Conditioned Facial Motioncited by 15
  16. 2024Exploring Self-Supervised Vision Transformers for Deepfake Detection: A Comparative Analysiscited by 15
  17. 2022M2TS: Multi-Scale Multi-Modal Approach Based on Transformer for Source Code Summarizationcited by 14
  18. 2023Securing Deep Generative Models with Universal Adversarial Signaturecited by 14
  19. 2022A Review of Deep Learning-based Approaches for Deepfake Content Detectioncited by 13
  20. 2024A Survey of Defenses Against AI-Generated Visual Media: Detection,Disruption, and Authenticationcited by 13
  21. 2024GenFace: A Large-Scale Fine-Grained Face Forgery Benchmark and Cross Appearance-Edge Learningcited by 12
  22. 2021FT-TDR: Frequency-guided Transformer and Top-Down Refinement Network for Blind Face Inpaintingcited by 11
  23. 2023Learning Pairwise Interaction for Generalizable DeepFake Detectioncited by 11
  24. 2023Multi-spectral Class Center Network for Face Manipulation Detection and Localizationcited by 11
  25. 2024MFCLIP: Multi-modal Fine-grained CLIP for Generalizable Diffusion Face Forgery Detectioncited by 11
  26. 2022Generalizable Deepfake Detection with Phase-Based Motion Analysiscited by 10
  27. 2023GenAI Mirage: The Impostor Bias and the Deepfake Detection Challenge in the Era of Artificial Illusionscited by 10
  28. 2024A Timely Survey on Vision Transformer for Deepfake Detectioncited by 10
  29. 2024Deepfake Media Generation and Detection in the Generative AI Era: A Survey and Outlookcited by 10
  30. 2021Detection of Deepfake Videos Using Long Distance Attentioncited by 9
  31. 2022Hierarchical Forgery Classifier On Multi-modality Face Forgery Cluescited by 9
  32. 2025Face-LLaVA: Facial Expression and Attribute Understanding through Instruction Tuningcited by 9
  33. 2024Mixture-of-Noises Enhanced Forgery-Aware Predictor for Multi-Face Manipulation Detection and Localizationcited by 7
  34. 2024Open-Set Deepfake Detection: A Parameter-Efficient Adaptation Method with Forgery Style Mixturecited by 7
  35. 2022Video Manipulations Beyond Faces: A Dataset with Human-Machine Analysiscited by 6
  36. 2022Multimodal Graph Learning for Deepfake Detectioncited by 6
  37. 2023Open Set Classification of GAN-based Image Manipulations via a ViT-based Hybrid Architecturecited by 6
  38. 2023Self-Supervised Graph Transformer for Deepfake Detectioncited by 6

50 most cited of 114 All 114 (JSON)