mlsanityData to 7 Oct 2026

Physiological Signals as a Forensic Modality for Talking-Face Deepfake Detection

arXiv 2607.21776 ↗First posted 23 Jul 2026cs.LG cs.CR cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Talking-face (TF) deepfake generation synthesizes photore- alistic facial video from a static source image and an au- dio signal, producing forgeries that current image-based detectors consistently fail to identify. Unlike face-swap ma- nipulation, TF synthesis has no underlying real video from which to inherit physiological characteristics, making re- mote photoplethysmography (rPPG) a uniquely motivated detection modality for this forgery category. We propose a detection framework that extracts per-video rPPG wave- forms via RhythmFormer and trains a suite of lightweight classifiers to distinguish real from synthesized physiologi- cal signals. Evaluated on the TF subset of Celeb-DF++ un- der a strict subject-independent protocol, where test identi- ties are completely separated from training identities, our 1D ResNet achieves an AUC of 0.806 and EER of 27.8%, placing it within 2.4 points of the best published general- purpose detector (Effort, ICML 2025) while operating ex- clusively on the physiological channel. We document a con- trolled reproduction study of DeepFakesON-Phys, the rep- resentative prior rPPG detector, demonstrating degrada- tion from AUC 0.999 on legacy face-swap data to 0.622 on the TF subset of Celeb-DF++. We further show that detec- tion difficulty is strongly method-dependent: AUC ranges from 0.985 (Real3DPortrait) to 0.690 (IP-LAP) across the seven TF generators, with the ranking remaining perfectly stable across all evaluation protocols. This spread reflects an interpretable physiological property of each generator rather than evaluation noise, and constitutes the primary theoretical contribution of the work.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 25 references that are in the map. First posted on arXiv 23 Jul 2026.

Builds on

19 of 25 references.

References · 19

  1. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  2. 2022SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animationcited by 380
  3. 2020Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detectioncited by 286
  4. 2024EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditionscited by 196
  5. 2019FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signalscited by 143
  6. 2019Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networkscited by 126
  7. 2024Orthogonal Subspace Decomposition for Generalizable AI-Generated Image Detectioncited by 102
  8. 2023Identity-Preserving Talking Face Generation with Landmark and Appearance Priorscited by 97
  9. 2024Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesiscited by 97
  10. 2024EDTalk: Efficient Disentanglement for Emotional Talking Head Synthesiscited by 79
  11. 2024AniTalker: Animate Vivid and Diverse Talking Faces through Identity-Decoupled Facial Motion Encodingcited by 67
  12. 2020DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythmscited by 65
7 more
  1. 2023PhysFormer++: Facial Video-based Physiological Measurement with SlowFast Temporal Difference Transformercited by 46
  2. 2024FLOAT: Generative Motion Latent Flow Matching for Audio-driven Talking Portraitcited by 40
  3. 2024Lips Are Lying: Spotting the Temporal Inconsistency between Audio and Visual in Lip-Syncing DeepFakescited by 32
  4. 2020DeepFakesON-Phys: DeepFakes Detection based on Heart Rate Estimationcited by 28
  5. 2024RhythmFormer: Extracting Patterned rPPG Signals based on Periodic Sparse Attentioncited by 28
  6. 2025Celeb-DF++: A Large-scale Challenging Video DeepFake Benchmark for Generalizable Forensicscited by 18
  7. 2025Detecting Lip-Syncing Deepfakes: Vision Temporal Transformer for Analyzing Mouth Inconsistenciescited by 7

Built on by

Not cited yet.