BioLip: Language-Generalizable Lip-Sync Deepfake Detection via Biomechanical Constraint Violation Modeling
Not on the map. Why?
Abstract (arXiv)
Existing lip-sync deepfake detectors rely on pixel artifacts or audio-visual correspondence, and both fail under generator or language shift because the features they learn are tied to the training distribution. We take a different approach. Authentic lip motion is constrained by tissue mechanics and neuromuscular bandwidth; current generators typically do not impose these constraints, producing trajectories with elevated variance in velocity, acceleration, and jerk that real speech does not exhibit. We exploit this signal, which we term temporal lip jitter, by computing kinematic statistics from 64 perioral landmarks over short sliding windows and feeding them into a lightweight three-branch network. The model uses only landmark coordinates: no pixels, no audio, and no voiceprint data. We train only on English data and test in a zero-shot setting on five unseen generators and seven languages.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 30 references that are in the map. First posted on arXiv 18 Apr 2026.
Builds on
22 of 30 references.
References · 22
- 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
- 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 669
- 2019MediaPipe: A Framework for Building Perception Pipelinescited by 572
- 2018MesoNet: a Compact Facial Video Forgery Detection Networkcited by 409
- 2018Exposing DeepFake Videos By Detecting Face Warping Artifactscited by 318
- 2020Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detectioncited by 286
- 2021FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Datasetcited by 179
- 2021Exploring Temporal Coherence for More General Video Face Forgery Detectioncited by 158
- 2022Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detectioncited by 130
- 2022VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wildcited by 106
- 2023Seeing What You Said: Talking Face Generation Guided by a Lip Reading Expertcited by 98
- 2023Identity-Preserving Talking Face Generation with Landmark and Appearance Priorscited by 97
10 more
- 2021Exploiting Fine-grained Face Forgery Clues via Progressive Enhancement Learningcited by 60
- 2024AVFF: Audio-Visual Feature Fusion for Video Deepfake Detectioncited by 59
- 2021Improving the Efficiency and Robustness of Deepfakes Detection through Precise Geometric Featurescited by 57
- 2023Diff2Lip: Audio Conditioned Diffusion Models for Lip-Synchronizationcited by 49
- 2024Lips Are Lying: Spotting the Temporal Inconsistency between Audio and Visual in Lip-Syncing DeepFakescited by 32
- 2024PolyGlotFake: A Novel Multilingual and Multimodal DeepFake Datasetcited by 16
- 2024Exposing Lip-syncing Deepfakes from Mouth Inconsistenciescited by 15
- 2025Detecting Lip-Syncing Deepfakes: Vision Temporal Transformer for Analyzing Mouth Inconsistenciescited by 7
- 2025Tell me Habibi, is it Real or Fake?cited by 6
- 2025PIA: Deepfake Detection Using Phoneme-Temporal and Identity-Dynamic Analysiscited by 5
Built on by
Not cited yet.