mlsanityData to 7 Oct 2026

LOKI: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Models

arXiv 2410.09732 ↗First posted 13 Oct 2024cs.CVcited by 51

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

With the rapid development of AI-generated content, the future internet may be inundated with synthetic data, making the discrimination of authentic and credible multimodal data increasingly challenging. Synthetic data detection has thus garnered widespread attention, and the performance of large multimodal models (LMMs) in this task has attracted significant interest. LMMs can provide natural language explanations for their authenticity judgments, enhancing the explainability of synthetic content detection. Simultaneously, the task of distinguishing between real and synthetic data effectively tests the perception, knowledge, and reasoning capabilities of LMMs. In response, we introduce LOKI, a novel benchmark designed to evaluate the ability of LMMs to detect synthetic data across multiple modalities. LOKI encompasses video, image, 3D, text, and audio modalities, comprising 18K carefully curated questions across 26 subcategories with clear difficulty levels. The benchmark includes coarse-grained judgment and multiple-choice questions, as well as fine-grained anomaly selection and explanation tasks, allowing for a comprehensive analysis of LMMs. We evaluated 22 open-source LMMs and 6 closed-source models on LOKI, highlighting their potential as synthetic data detectors and also revealing some limitations in the development of LMM capabilities. More information about LOKI can be found at https://opendatalab.github.io/LOKI/

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 50 of its 148 references that are in the map. First posted on arXiv 13 Oct 2024.

Builds on

50 of 148 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Models20 Apr

Other references · 49

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Denoising Diffusion Implicit Modelscited by 9,655
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  6. 2023Mistral 7Bcited by 3,065
  7. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  8. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  9. 2022Emergent Abilities of Large Language Modelscited by 2,483
  10. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  11. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  12. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
37 more
  1. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  2. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  3. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  4. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
  5. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
  6. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  7. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 592
  8. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 591
  9. 2023Textbooks Are All You Needcited by 561
  10. 2023How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detectioncited by 472
  11. 2023MusicLM: Generating Music From Textcited by 442
  12. 2023Photorealistic Video Generation with Diffusion Modelscited by 304
  13. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  14. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  15. 2016Skin Lesion Analysis toward Melanoma Detection: A Challenge at the International Symposium on Biomedical Imaging (ISBI) 2016, hosted by the International Skin Imaging Collaboration (ISIC)cited by 197
  16. 2023VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluationcited by 196
  17. 2024A Sanity Check for AI-generated Image Detectioncited by 187
  18. 2024DreamBench++: A Human-Aligned Benchmark for Personalized Image Generationcited by 131
  19. 2024NVLM: Open Frontier-Class Multimodal LLMscited by 124
  20. 2023Investigating the Catastrophic Forgetting in Multimodal Large Language Modelscited by 119
  21. 2024video-SALMONN: Speech-Enhanced Audio-Visual Large Language Modelscited by 101
  22. 2023Ghostbuster: Detecting Text Ghostwritten by Large Language Modelscited by 90
  23. 2024Model Collapse Demystified: The Case of Regressioncited by 80
  24. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspectivecited by 73
  25. 2024Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verificationcited by 43
  26. 2023Nepotistically Trained Generative-AI Models Collapsecited by 27
  27. 2024As Good As A Coin Toss: Human detection of AI-generated images, videos, audio, and audiovisual stimulicited by 22
  28. 2024Distinguish Any Fake Videos: Unleashing the Power of Large-scale Data and Motion Featurescited by 20
  29. 2023Foley Sound Synthesis at the DCASE 2023 Challengecited by 19
  30. 2024CtrSVDD: A Benchmark Dataset and Baseline Analysis for Controlled Singing Voice Deepfake Detectioncited by 16
  31. 2019Unsupervised Singing Voice Conversioncited by 14
  32. 2023M3Dsynth: A dataset of medical 3D images with AI-generated local manipulationscited by 11
  33. 2024Leveraging BEV Paradigm for Ground-to-Aerial Image Synthesiscited by 11
  34. 2024How Do Neural Spoofing Countermeasures Detect Partially Spoofed Audio?cited by 10
  35. 2024Efficient Pretraining Data Selection for Language Models via Multi-Actor Collaborationcited by 10
  36. 2024GradeADreamer: Enhanced Text-to-3D Generation Using Gaussian Splatting and Multi-View Diffusioncited by 4
  37. 2019ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speechcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 26; 2026: 24.

51 papers.

Sort:

By area:

  1. 2025Echo-4o: Harnessing the Power of GPT-4o Synthetic Images for Improved Image Generationcited by 95
  2. 2025Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection with Artifact Explanationcited by 66
  3. 2025LEGION: Learning to Ground and Explain for Synthetic Image Detectioncited by 65
  4. 2025GPT-ImgEval: A Comprehensive Benchmark for Diagnosing GPT4o in Image Generationcited by 57
  5. 2026Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perceptioncited by 47
  6. 2024Where am I? Cross-View Geo-localization with Natural Language Descriptionscited by 38
  7. 2025Ivy-Fake: A Unified Explainable Framework and Benchmark for Image and Video AIGC Detectioncited by 34
  8. 2025AIGI-Holmes: Towards Explainable and Generalizable AI-Generated Image Detection via Multimodal Large Language Modelscited by 33
  9. 2025Veritas: Generalizable Deepfake Detection via Pattern-Aware Reasoningcited by 23
  10. 2025Toward Generalizable Forgery Detection and Reasoningcited by 20
  11. 2025Impossible Videoscited by 16
  12. 2025Survey on AI-Generated Media Detection: From Non-MLLM to MLLMcited by 12
38 more
  1. 2025FakeScope: Large Multimodal Expert Model for Transparent AI-Generated Image Forensicscited by 11
  2. 2025RealGen: Photorealistic Text-to-Image Generation via Detector-Guided Rewardscited by 11
  3. 2026VideoVeritas: AI-Generated Video Detection via Perception Pretext Reinforcement Learningcited by 11
  4. 2025SAGI: Semantically Aligned and Uncertainty Guided AI Image Inpaintingcited by 10
  5. 2026GenClaw: Code-Driven Agentic Image Generationcited by 10
  6. 2025BLINK-Twice: You see, but do you observe? A Reasoning Benchmark on Visual Perceptioncited by 9
  7. 2026Fake-HR1: Rethinking Reasoning of Vision Language Model for Synthetic Image Detectioncited by 8
  8. 2025Explainable Synthetic Image Detection through Diffusion Timestep Ensemblingcited by 7
  9. 2025Guard Me If You Know Me: Protecting Specific Face-Identity from Deepfakescited by 7
  10. 2025ThinkFake: Reasoning in Multimodal Large Language Models for AI-Generated Image Detectioncited by 7
  11. 2025UrbanFeel: A Comprehensive Benchmark for Temporal and Perceptual Understanding of City Scenes through Human Perspectivecited by 7
  12. 2026EvoGuard: An Extensible Agentic RL-based Framework for Practical and Evolving AI-Generated Image Detectioncited by 6
  13. 2025MajutsuCity: Language-driven Aesthetic-adaptive City Generation with Controllable 3D Assets and Layoutscited by 5
  14. 2025REVEAL: Reasoning-Enhanced Forensic Evidence Analysis for Explainable AI-Generated Image Detectioncited by 4
  15. 2025TriDF: Evaluating Perception, Detection, and Hallucination for Interpretable DeepFake Detectioncited by 4
  16. 2025Code-in-the-Loop Forensics: Agentic Tool Use for Image Forgery Detectioncited by 4
  17. 2025RobustSora: De-Watermarked Benchmark for Robust AI-Generated Video Detectioncited by 3
  18. 2026UniGenDet: A Unified Generative-Discriminative Framework for Co-Evolutionary Image Generation and Generated Image Detectioncited by 3
  19. 2026FakeVLM-R1: Internalizing Physical Laws via CoT for Synthetic Image Detectioncited by 3
  20. 2025Virus Infection Attack on LLMs: Your Poisoning Can Spread "VIA" Synthetic Datacited by 2
  21. 2025VideoASMR-Bench: Can AI-Generated ASMR Videos Fool VLMs and Humans?cited by 2
  22. 2026Omni-Fake: Benchmarking Unified Multimodal Social Media Deepfake Detectioncited by 2
  23. 2025AVFakeBench: A Comprehensive Audio-Video Forgery Detection Benchmark for AV-LMMscited by 1
  24. 2026See and Fix the Flaws: Enabling VLMs and Diffusion Models to Comprehend Visual Artifacts via Agentic Data Synthesiscited by 1
  25. 2026GenVideoLens: Where LVLMs Fall Short in AI-Generated Video Detection?cited by 1
  26. 2026EditSleuth: A Dataset of Grounded Reasoning Chains for Image-Edit Forensicscited by 1
  27. 2026Video as Natural Augmentation: Towards Unified AI-Generated Image and Video Detectioncited by 1
  28. 2026SalArt-VQA: Diagnosing Whether VLMs Understand Salient Artifacts in Generated Imagescited by 1
  29. 2026SafeGuard: A Multi-Agent Perception-Reasoning Framework for Social-Risk AI-Generated Video Detectioncited by 1
  30. 2026SynMulti: Synthetic-to-Real Learning for Multimodal Video Understandingcited by 0
  31. 2026OmniVL-Guard Pro: A Tool-Augmented Agent for Omnibus Vision-Language Forensicscited by 0
  32. 2026When Eyes Betray AI: Social Gaze Consistency as a Semantic Cue for AI-Generated Image Detectioncited by 0
  33. 2026Dissect and Prune: Enhancing Robustness in AI-Generated Image Detectioncited by 0
  34. 2026Frontier vision-language models have overtaken young adults at detecting AI-generated portraits -- but not their calibrationcited by 0
  35. 2026Editable Visual Designcited by 0
  36. 2026SynDORBench: Evaluating LVLM Perceptual Robustness Under Physically Constrained Visibility Conditionscited by 0
  37. 2026Mandela-Bench: Multimodal Models Remember Canonical Images Instead of Seeing Themcited by 0
  38. 2026IVT-Guard: All-in-One Reasoning Model for AI-Generated Content Detectioncited by 0

50 most cited of 51 All 51 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Multimodal Misinformation Detection by Learning from Synthetic Data with Multimodal LLMs
  2. 2025HintsOfTruth: A Multimodal Checkworthiness Detection Dataset with Real and Synthetic Claims
  3. 2026Can Multimodal Large Language Models Generate and Detect Multimodal Social Media Fake News?
  4. 2026The Synthetic Media Shift: Tracking the Rise, Virality, and Detectability of AI-Generated Multimodal Misinformation
  5. 2026The Paradigm Shift: A Comprehensive Survey on Large Vision Language Models for Multimodal Fake News Detection
  6. 2025AI-GenBench: A New Ongoing Benchmark for AI-Generated Image Detection
  7. 2025Learning from Reasoning Failures via Synthetic Data Generation
  8. 2026BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models