mlsanityData to 7 Oct 2026

Inconsistency-aware Multimodal Schr"odinger Bridge for Deepfake Localization

arXiv 2605.23113 ↗First posted 22 May 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Audio-visual deepfake localization demands interval-level outputs that serve as temporal evidence. Despite recent progress, symmetric fusion under single-sided or asynchronous forgeries propagates cross-modal noise, degrading high-precision localization. We present IaMSB, an inconsistency-aware multimodal Schr"odinger Bridge (SB) that jointly estimates cross-modal consistency and performs interval-level localization. Unlike diffusion models, SB minimizes path-distribution discrepancy and yields consistency scores without explicit noise injection or denoising. With the Schr"odinger Bridge (SB), IaMSB unifies consistency estimation, cross-modal information selection, and bridge-step scheduling in one framework. Specifically, a lightweight coarse bridge first proposes candidate intervals and estimates cross-modal consistency; these statistics select cross-modal witness signals and allocate bridge steps asymmetrically across modalities. A refinement bridge then performs step-tuned fusion and outputs refined, time-aligned intervals. IaMSB anticipates single-sided and asynchronous forgeries and, using bottlenecked cross-modal interaction with step allocation, suppresses noise transfer, avoids unnecessary iterations. Across benchmarks, IaMSB stabilizes strict-IoU boundary precision, raising [email protected] by 3%~10%, and yields improved high-precision localization, particularly for single-sided forgeries.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 41 references that are in the map. First posted on arXiv 22 May 2026.

Builds on

31 of 41 references.

References · 31

  1. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  2. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  3. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modelingcited by 594
  4. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 308
  5. 2022Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Predictioncited by 277
  6. 2020The Creation and Detection of Deepfakes: A Surveycited by 246
  7. 2020Media Forensics and DeepFakes: an overviewcited by 208
  8. 2022Contrastive Audio-Visual Masked Autoencodercited by 142
  9. 2020Not made for each other- Audio-Visual Dissonance-based Deepfake Detection and Localizationcited by 89
  10. 2024Deep Multimodal Learning with Missing Modality: A Surveycited by 89
  11. 2023AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Datasetcited by 79
  12. 2023Zero-TPrune: Zero-Shot Token Pruning through Leveraging of the Attention Graph in Pre-Trained Transformerscited by 61
19 more
  1. 2024AVFF: Audio-Visual Feature Fusion for Video Deepfake Detectioncited by 59
  2. 2022MAViL: Masked Audio-Video Learnerscited by 56
  3. 2024MADTP: Multimodal Alignment-Guided Dynamic Token Pruning for Accelerating Vision-Language Transformercited by 53
  4. 2023Glitch in the Matrix: A Large Scale Benchmark for Content Driven Audio-Visual Forgery Detection and Localizationcited by 42
  5. 2024Diagnosing and Re-learning for Balanced Multimodal Learningcited by 40
  6. 2023UMMAFormer: A Universal Multimodal-adaptive Transformer Framework for Temporal Forgery Localizationcited by 33
  7. 2024Lips Are Lying: Spotting the Temporal Inconsistency between Audio and Visual in Lip-Syncing DeepFakescited by 32
  8. 2024HiCMAE: Hierarchical Contrastive Masked Autoencoder for Self-Supervised Audio-Visual Emotion Recognitioncited by 30
  9. 2025CAV-MAE Sync: Improving Contrastive Audio-Visual Mask Autoencoders via Fine-Grained Alignmentcited by 14
  10. 2024Reflected Schr"odinger Bridge for Constrained Generative Modelingcited by 12
  11. 2024CEEBERT: Cross-Domain Inference in Early Exit BERTcited by 12
  12. 2024DiMoDif: Discourse Modality-information Differentiation for Audio-visual Deepfake Detection and Localizationcited by 12
  13. 2023Boundary-Denoising for Video Activity Localizationcited by 10
  14. 2024Coarse-to-Fine Proposal Refinement Framework for Audio Temporal Forgery Detection and Localizationcited by 9
  15. 2024Contextual Cross-Modal Attention for Audio-Visual Deepfake Detection and Localizationcited by 6
  16. 2023Rethink Cross-Modal Fusion in Weakly-Supervised Audio-Visual Video Parsingcited by 5
  17. 2024Faster Diffusion Action Segmentationcited by 2
  18. 2025Language-Guided Contrastive Audio-Visual Masked Autoencoder with Automatically Generated Audio-Visual-Text Triplets from Videoscited by 2
  19. 2025Generic Event Boundary Detection via Denoising Diffusioncited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Geometry-based Schr"odinger Bridges for Trustworthy Multimodal Fusion
  2. 2025Multi-modal Deepfake Detection and Localization with FPN-Transformer
  3. 2026Audio-Anchored Fusion of Multi-Ratio DiT Reconstruction Residuals for Cross-Domain Audio Deepfake Detection
  4. 2025Schr"odinger Bridge Consistency Trajectory Models for Speech Enhancement
  5. 2026X-AVDT: Audio-Visual Cross-Attention for Robust Deepfake Detection
  6. 2026Rectified Schr"odinger Bridge Matching for Few-Step Visual Navigation
  7. 2026ToolDF: Tool-Integrated Reasoning for Mixed-Authenticity Audio Deepfake Detection
  8. 2026MADBench: A Benchmark for Modality-Aware Audio Deepfake Detection