# Exposing Lip-syncing Deepfakes from Mouth Inconsistencies

arXiv: https://arxiv.org/abs/2401.10113 · HTML: https://mlsanity.com/paper/2401.10113/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 36 references that are in the map.
- First posted on arXiv 18 Jan 2024.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild](https://mlsanity.com/paper/2008.10010/) (2020)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [Deep Audio-Visual Speech Recognition](https://mlsanity.com/paper/1809.02108/) (2018)
- [FSGAN: Subject Agnostic Face Swapping and Reenactment](https://mlsanity.com/paper/1908.05932/) (2019)
- [Exposing DeepFake Videos By Detecting Face Warping Artifacts](https://mlsanity.com/paper/1811.00656/) (2018)
- [Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection](https://mlsanity.com/paper/2012.07657/) (2020)
- [Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis](https://mlsanity.com/paper/1806.04558/) (2018)
- [FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset](https://mlsanity.com/paper/2108.05080/) (2021)
- [Exploring Temporal Coherence for More General Video Face Forgery Detection](https://mlsanity.com/paper/2108.06693/) (2021)
- [DeepFaceLab: Integrated, flexible and extensible face-swapping framework](https://mlsanity.com/paper/2005.05535/) (2020)
- [Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection](https://mlsanity.com/paper/2201.07131/) (2022)
- [Fast Face-swap Using Convolutional Neural Networks](https://mlsanity.com/paper/1611.09577/) (2016)
- [Deep Learning for Deepfakes Creation and Detection: A Survey](https://mlsanity.com/paper/1909.11573/) (2019)
- [Audio-driven Talking Face Video Generation with Learning-based Personalized Head Pose](https://mlsanity.com/paper/2002.10137/) (2020)
- [Self-Supervised Video Forensics by Audio-Visual Anomaly Detection](https://mlsanity.com/paper/2301.01767/) (2023)
- [KoDF: A Large-scale Korean DeepFake Detection Dataset](https://mlsanity.com/paper/2103.10094/) (2021)
- [DeepFake Detection Based on the Discrepancy Between the Face and its Context](https://mlsanity.com/paper/2008.12262/) (2020)
- [GANprintR: Improved Fakes and Evaluation of the State of the Art in Face Manipulation Detection](https://mlsanity.com/paper/1911.05351/) (2019)
- [DeepFakes Evolution: Analysis of Facial Regions and Fake Detection Performance](https://mlsanity.com/paper/2004.07532/) (2020)
- [FNeVR: Neural Volume Rendering for Face Animation](https://mlsanity.com/paper/2209.10340/) (2022)
- [Watch Those Words: Video Falsification Detection Using Word-Conditioned Facial Motion](https://mlsanity.com/paper/2112.10936/) (2021)
- [Integrating Audio-Visual Features for Multimodal Deepfake Detection](https://mlsanity.com/paper/2310.03827/) (2023)
- [Unsupervised Multimodal Deepfake Detection Using Intra- and Cross-Modal Inconsistencies](https://mlsanity.com/paper/2311.17088/) (2023)

## Built on by (most cited first, 15 of 15)

- [Detecting Lip-Syncing Deepfakes: Vision Temporal Transformer for Analyzing Mouth Inconsistencies](https://mlsanity.com/paper/2504.01470/) (2025)
- [KLASSify to Verify: Audio-Visual Deepfake Detection Using SSL-based Audio and Handcrafted Visual Features](https://mlsanity.com/paper/2508.07337/) (2025)
- [PIA: Deepfake Detection Using Phoneme-Temporal and Identity-Dynamic Analysis](https://mlsanity.com/paper/2510.14241/) (2025)
- [Human Action CLIPs: Detecting AI-generated Human Motion](https://mlsanity.com/paper/2412.00526/) (2024)
- [DeepFake-O-Meter v2.0: An Open Platform for DeepFake Detection](https://mlsanity.com/paper/2404.13146/) (2024)
- [Pindrop it! Audio and Visual Deepfake Countermeasures for Robust Detection and Fine Grained-Localization](https://mlsanity.com/paper/2508.08141/) (2025)
- [Omni-Fake: Benchmarking Unified Multimodal Social Media Deepfake Detection](https://mlsanity.com/paper/2605.01638/) (2026)
- [GC-ConsFlow: Leveraging Optical Flow Residuals and Global Context for Robust Deepfake Detection](https://mlsanity.com/paper/2501.13435/) (2025)
- [Deepfake Forensic Analysis: Source Dataset Attribution and Legal Implications of Synthetic Media Manipulation](https://mlsanity.com/paper/2505.11110/) (2025)
- [Video Forgery Detection with Optical Flow Residuals and Spatial-Temporal Consistency](https://mlsanity.com/paper/2508.00397/) (2025)
- [Zero-Shot Visual Deepfake Detection: Can AI Predict and Prevent Fake Content Before It's Created?](https://mlsanity.com/paper/2509.18461/) (2025)
- [Improving Lip-synchrony in Direct Audio-Visual Speech-to-Speech Translation](https://mlsanity.com/paper/2412.16530/) (2024)
- [Unmasking Synthetic Realities in Generative AI: A Comprehensive Review of Adversarially Robust Deepfake Detection Systems](https://mlsanity.com/paper/2507.21157/) (2025)
- [BioLip: Language-Generalizable Lip-Sync Deepfake Detection via Biomechanical Constraint Violation Modeling](https://mlsanity.com/paper/2604.16808/) (2026)
- [CAM-VFD: Cross-Attention Multimodal Video Forgery Detection](https://mlsanity.com/paper/2605.17133/) (2026)

## Abstract (arXiv)

A lip-syncing deepfake is a digitally manipulated video in which a person's lip movements are created convincingly using AI models to match altered or entirely new audio. Lip-syncing deepfakes are a dangerous type of deepfakes as the artifacts are limited to the lip region and more difficult to discern. In this paper, we describe a novel approach, LIP-syncing detection based on mouth INConsistency (LIPINC), for lip-syncing deepfake detection by identifying temporal inconsistencies in the mouth region. These inconsistencies are seen in the adjacent frames and throughout the video. Our model can successfully capture these irregularities and outperforms the state-of-the-art methods on several benchmark deepfake datasets. Code is available at https://github.com/skrantidatta/LIPINC