Omni-Fake: Benchmarking Unified Multimodal Social Media Deepfake Detection
Not on the map. Why?
Abstract (arXiv)
Multimodal deepfakes are proliferating on social media and threaten authenticity, information integrity, and digital forensics. Existing benchmarks are constrained by their single-modality scope, simplified manipulations, or unrealistic distributions, which limit their ability to assess real-world robustness. To address these limitations, we present Omni-Fake, a unified omni-dataset for comprehensive multimodal deepfake detection in social-media settings. It comprises Omni-Fake-Set, a large-scale, high-quality dataset with 1M+ samples, and Omni-Fake-OOD, an out-of-distribution benchmark with 200k+ samples intentionally excluded from training to evaluate generalization. Omni-Fake spans four modalities (image, audio, video, and audio-video talking head) and supports a joint detection-localization-explanation protocol. On top of Omni-Fake, we further propose Omni-Fake-R1, a reinforcement-learning-driven multimodal detector that adaptively integrates visual and auditory cues and outputs structured decisions, localization, and natural-language explanations. Extensive experiments show significant gains in detection accuracy, cross-modal generalization, and explainability over state-of-the-art baselines. Project page: https://tianxiao1201.github.io/omni-fake-project-page/
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 82 of its 97 references that are in the map. First posted on arXiv 2 May 2026.
Builds on
82 of 97 references.
References · 82
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2024GPT-4o System Cardcited by 4,467
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
- 2021Alias-Free Generative Adversarial Networkscited by 1,181
38 more
- 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
- 2019CNN-generated images are surprisingly easy to spot... for nowcited by 842
- 2025Group Sequence Policy Optimizationcited by 788
- 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
- 2025Qwen2.5-Omni Technical Reportcited by 704
- 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
- 2023Towards Universal Fake Image Detectors that Generalize Across Generative Modelscited by 483
- 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 467
- 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 438
- 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 323
- 2023GenImage: A Million-Scale Benchmark for Detecting AI-Generated Imagecited by 318
- 2020Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detectioncited by 286
- 2022On the detection of synthetic images generated by diffusion modelscited by 282
- 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMscited by 276
- 2019On the Detection of Digital Face Manipulationcited by 259
- 2022CelebV-HQ: A Large-Scale Video Facial Attributes Datasetcited by 246
- 2024EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditionscited by 196
- 2024AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animationcited by 191
- 2021FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Datasetcited by 179
- 2025Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyondcited by 175
- 2021AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networkscited by 173
- 2022Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detectioncited by 130
- 2021ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysiscited by 124
- 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformercited by 121
- 2024FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Modelscited by 111
- 2024SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Modelcited by 107
- 2024One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videoscited by 99
- 2024Hallo2: Long-Duration and High-Resolution Audio-Driven Portrait Image Animationcited by 95
- 2024DeMamba: AI-Generated Video Detection on Million-Scale GenVideo Benchmarkcited by 85
- 2023Self-Supervised Video Forensics by Audio-Visual Anomaly Detectioncited by 84
- 2025VideoPainter: Any-length Video Inpainting and Editing with Plug-and-Play Context Controlcited by 83
- 2021KoDF: A Large-scale Korean DeepFake Detection Datasetcited by 80
- 2023AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Datasetcited by 79
- 2024EasyAnimate: High-Performance Video Generation Framework with Hybrid Windows Attention and Reward Backpropagationcited by 76
- 2024Deepfake Generation and Detection: A Benchmark and Surveycited by 66
- 2025Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection with Artifact Explanationcited by 66
- 2023Seeing is not always believing: Benchmarking Human and Model Perception of AI-Generated Imagescited by 65
- 2025LEGION: Learning to Ground and Explain for Synthetic Image Detectioncited by 65
50 most cited of 82 All 82 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.