mlsanityData to 14 Oct 2026

Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection

arXiv 2012.07657 ↗First posted 14 Dec 2020cs.CVcited by 287

Not on the map. Why?

Abstract (arXiv)

Although current deep learning-based face forgery detectors achieve impressive performance in constrained scenarios, they are vulnerable to samples created by unseen manipulation methods. Some recent works show improvements in generalisation but rely on cues that are easily corrupted by common post-processing operations such as compression. In this paper, we propose LipForensics, a detection approach capable of both generalising to novel manipulations and withstanding various distortions. LipForensics targets high-level semantic irregularities in mouth movements, which are common in many generated videos. It consists in first pretraining a spatio-temporal network to perform visual speech recognition (lipreading), thus learning rich internal representations related to natural mouth motion. A temporal network is subsequently finetuned on fixed mouth embeddings of real and forged data in order to detect fake videos based on mouth movements without overfitting to low-level, manipulation-specific artefacts. Extensive experiments show that this simple approach significantly surpasses the state-of-the-art in terms of generalisation to unseen manipulations and robustness to perturbations, as well as shed light on the factors responsible for its performance. Code is available on GitHub.

Key facts. Cited by 287 papers in the map (data through 14 Oct 2026). First posted on arXiv 14 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 7; 2022: 38; 2023: 56; 2024: 61; 2025: 75; 2026: 50.

287 papers.

Sort:

By area:

  1. 2022Detecting Deepfakes with Self-Blended Imagescited by 338
  2. 2022CelebV-HQ: A Large-Scale Video Facial Attributes Datasetcited by 250
  3. 2021FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Datasetcited by 181
  4. 2023UCF: Uncovering Common Features for Generalizable Deepfake Detectioncited by 180
  5. 2022Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detectioncited by 173
  6. 2021Exploring Temporal Coherence for More General Video Face Forgery Detectioncited by 159
  7. 2022Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detectioncited by 130
  8. 2024DF40: Toward Next-Generation Deepfake Detectioncited by 129
  9. 2023Transcending Forgery Specificity with Latent Space Augmentation for Generalizable Deepfake Detectioncited by 127
  10. 2022Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalizationcited by 123
  11. 2024C2P-CLIP: Injecting Category Common Prompt in CLIP to Enhance Generalization in Deepfake Detectioncited by 113
  12. 2024Orthogonal Subspace Decomposition for Generalizable AI-Generated Image Detectioncited by 105
38 more
  1. 2023TALL: Thumbnail Layout for Deepfake Video Detectioncited by 100
  2. 2022Visual Speech Recognition for Multiple Languages in the Wildcited by 95
  3. 2022Protecting Celebrities from DeepFake with Identity Consistency Transformercited by 95
  4. 2024LAA-Net: Localized Artifact Attention Network for Quality-Agnostic and Generalizable Deepfake Detectioncited by 89
  5. 2023Self-Supervised Video Forensics by Audio-Visual Anomaly Detectioncited by 85
  6. 2021Deepfakes Generation and Detection: State-of-the-art, open challenges, countermeasures, and way forwardcited by 82
  7. 2023Robustness of AI-Image Detectors: Fundamental Limits and Practical Attackscited by 81
  8. 2023AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Datasetcited by 80
  9. 2023AltFreezing for More General Video Face Forgery Detectioncited by 78
  10. 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 77
  11. 2022UIA-ViT: Unsupervised Inconsistency-Aware Method based on Vision Transformer for Face Forgery Detectioncited by 69
  12. 2025Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection with Artifact Explanationcited by 68
  13. 2024Deepfake Generation and Detection: A Benchmark and Surveycited by 67
  14. 2023BlendFace: Re-designing Identity Encoders for Face-Swappingcited by 66
  15. 2024Exploiting Style Latent Flows for Generalizing Deepfake Video Detectioncited by 59
  16. 2024AVFF: Audio-Visual Feature Fusion for Video Deepfake Detectioncited by 59
  17. 2024Generalizing Deepfake Video Detection with Plug-and-Play: Video-Level Blending and Spatiotemporal Adapter Tuningcited by 59
  18. 2021Countering Malicious DeepFakes: Survey, Battleground, and Horizoncited by 56
  19. 2023Detecting and Grounding Multi-Modal Media Manipulation and Beyondcited by 56
  20. 2022SeeABLE: Soft Discrepancies and Bounded Contrastive Learning for Exposing Deepfakescited by 49
  21. 2024Can We Leave Deepfake Data Behind in Training Deepfake Detector?cited by 49
  22. 2024FreqBlender: Enhancing DeepFake Detection by Blending Frequency Knowledgecited by 48
  23. 2022Deepfake Network Architecture Attributioncited by 47
  24. 2022Jointly Learning Visual and Auditory Speech Representations from Raw Datacited by 47
  25. 2024Exposing the Deception: Uncovering More Forgery Clues for Deepfake Detectioncited by 47
  26. 2023Towards General Visual-Linguistic Face Forgery Detectioncited by 45
  27. 2022Voice-Face Homogeneity Tells Deepfakecited by 44
  28. 2022Audio-Visual Person-of-Interest DeepFake Detectioncited by 43
  29. 2022Delving into Sequential Patches for Deepfake Detectioncited by 41
  30. 2024SHIELD : An Evaluation Benchmark for Face Spoofing and Forgery Detection with Multimodal Large Language Modelscited by 40
  31. 2024On Learning Multi-Modal Forgery Representation for Diffusion Generated Video Detectioncited by 40
  32. 2025Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detectorcited by 40
  33. 2024Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Modelcited by 38
  34. 2022Exploring Disentangled Content Information for Face Forgery Detectioncited by 36
  35. 2022Learning Second Order Local Anomaly for General Face Forgery Detectioncited by 36
  36. 2023Robustness and Generalizability of Deepfake Detection: A Study with Diffusion Modelscited by 36
  37. 2024Poisoned Forgery Face: Towards Backdoor Attacks on Face Forgery Detectioncited by 35
  38. 2024X2-DFD: A framework for eXplainable and eXtendable Deepfake Detectioncited by 35

50 most cited of 287 All 287 (JSON)