mlsanityData to 14 Oct 2026

DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Experts

arXiv 2412.10510 ↗First posted 13 Dec 2024cs.CV cs.CLcited by 43

Not on the map. Why?

Abstract (arXiv)

The proliferation of disinformation demands reliable and scalable fact-checking solutions. We present Dynamic Evidence-based FAct-checking with Multimodal Experts (DEFAME), a modular, zero-shot MLLM pipeline for open-domain, text-image claim verification. DEFAME operates in a six-stage process, dynamically selecting the tools and search depth to extract and evaluate textual and visual evidence. Unlike prior approaches that are text-only, lack explainability, or rely solely on parametric knowledge, DEFAME performs end-to-end verification, accounting for images in claims and evidence while generating structured, multimodal reports. Evaluation on the popular benchmarks VERITE, AVerITeC, and MOCHEG shows that DEFAME surpasses all previous methods, establishing itself as the new state-of-the-art fact-checking system for uni- and multimodal fact-checking. Moreover, we introduce a new multimodal benchmark, ClaimReview2024+, featuring claims after the knowledge cutoff of GPT-4o, avoiding data leakage. Here, DEFAME drastically outperforms the GPT-4o baselines, showing temporal generalizability and the potential for real-time fact-checking.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). Builds on 64 of its 92 references that are in the map. First posted on arXiv 13 Dec 2024.

Builds on

64 of 92 references.

References · 64

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  3. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  4. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
  5. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
  6. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  7. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 603
  8. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  9. 2017"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detectioncited by 363
  10. 2023GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localizationcited by 187
  11. 2023Can LLM-Generated Misinformation Be Detected?cited by 177
  12. 2023Fact-Checking Complex Claims with Program-Guided Reasoningcited by 146
38 more
  1. 2023AVeriTeC: A Dataset for Real-world Claim Verification with Evidence from the Webcited by 144
  2. 2020Generating Fact Checking Explanationscited by 132
  3. 2022TruFor: Leveraging all-round clues for trustworthy image forgery detection and localizationcited by 130
  4. 2021NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Mediacited by 107
  5. 2022End-to-End Multimodal Fact-Checking and Explanation Generation: A Challenging Dataset and Modelscited by 107
  6. 2023Detecting and Grounding Multi-Modal Media Manipulationcited by 99
  7. 2023Task Contamination: Language Models May Not Be Few-Shot Anymorecited by 99
  8. 2023Towards LLM-based Fact Verification on News Claims with a Hierarchical Step-by-Step Prompting Methodcited by 97
  9. 2023Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkerscited by 87
  10. 2024SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detectioncited by 74
  11. 2021Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online Resourcescited by 71
  12. 2024Can ChatGPT Detect DeepFakes? A Study of Using Multimodal Large Language Models for Media Forensicscited by 69
  13. 2022MuMiN: A Large-Scale Multilingual Multimodal Fact-Checked Misinformation Social Network Datasetcited by 61
  14. 2024MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMscited by 61
  15. 2023Multimodal Automated Fact-Checking: A Surveycited by 53
  16. 2019Fact-Checking Meets Fauxtography: Verifying Claims About Imagescited by 52
  17. 2018On the Origins of Memes by Means of Fringe Web Communitiescited by 47
  18. 2023Benchmarks for Automated Commonsense Reasoning: A Surveycited by 40
  19. 2023VERITE: A Robust Benchmark for Multimodal Misinformation Detection Accounting for Unimodal Biascited by 39
  20. 2021AmbiFC: Fact-Checking Ambiguous Claims with Evidencecited by 38
  21. 2023FactLLaMA: Optimizing Instruction-Following Language Models with External Knowledge for Automated Fact-Checkingcited by 37
  22. 2023Reading and Reasoning over Chart Images for Evidence-based Automated Fact-Checkingcited by 35
  23. 2023Charting New Territories: Exploring the Geographic and Geospatial Capabilities of Multimodal LLMscited by 34
  24. 2020Multimodal Analytics for Real-world News using Measures of Cross-modal Entity Consistencycited by 33
  25. 2018Deep Multimodal Image-Repurposing Detectioncited by 31
  26. 2023Self-Checker: Plug-and-Play Modules for Fact-Checking with Large Language Modelscited by 31
  27. 2024The Automated Verification of Textual Claims (AVeriTeC) Shared Taskcited by 29
  28. 2017Multimedia Semantic Integrity Assessment Using Joint Embedding Of Images And Textcited by 28
  29. 2024OpenFactCheck: A Unified Framework for Factuality Evaluation of LLMscited by 28
  30. 2024RAGAR, Your Falsehood Radar: RAG-Augmented Reasoning for Political Fact-Checking using Multimodal Large Language Modelscited by 27
  31. 2024AMMeBa: A Large-Scale Survey and Dataset of Media-Based Misinformation In-The-Wildcited by 27
  32. 2023ChartCheck: Explainable Fact-Checking over Real-World Chart Imagescited by 26
  33. 2024Claim Verification in the Age of Large Language Models: A Surveycited by 24
  34. 2024Multimodal Misinformation Detection using Large Vision-Language Modelscited by 21
  35. 2024Generative Large Language Models in Automated Fact-Checking: A Surveycited by 19
  36. 2023RED-DOT: Multimodal Fact-checking via Relevant Evidence Detectioncited by 18
  37. 2024Re-Search for The Truth: Multi-round Retrieval-augmented Large Language Models are Strong Fake News Detectorscited by 16
  38. 2024Improving Faithfulness of Large Language Models in Summarization via Sliding Generation and Self-Consistencycited by 14

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 19; 2026: 24.

43 papers.

Sort:

By area:

  1. 2025AVerImaTeC: A Dataset for Automatic Verification of Image-Text Claims with Evidence from the Webcited by 18
  2. 2025COVE: COntext and VEracity prediction for out-of-context imagescited by 14
  3. 2025Fact-R1: Towards Explainable Video Misinformation Detection with Deep Reasoningcited by 14
  4. 2025M4FC: a Multimodal, Multilingual, Multicultural, Multitask Real-World Fact-Checking Datasetcited by 7
  5. 2025Debating Truth: Debate-driven Claim Verification with Multiple Large Language Model Agentscited by 6
  6. 2025Drifting Away from Truth: GenAI-Driven News Diversity Challenges LVLM-Based Misinformation Detectioncited by 6
  7. 2026VeriTaS: The First Dynamic Benchmark for Multimodal Automated Fact-Checkingcited by 6
  8. 2025Seeing Through Deception: Uncovering Misleading Creator Intent in Multimodal News with Vision-Language Modelscited by 5
  9. 2025Adversarial Attacks Against Automated Fact-Checking: A Surveycited by 5
  10. 2025T^2Agent A Tool-augmented Multimodal Misinformation Detection Agent with Monte Carlo Tree Searchcited by 4
  11. 2025Multimedia Verification Through Multi-Agent Deep Research Multimodal Large Language Modelscited by 4
  12. 2025Fact2Fiction: Targeted Poisoning Attack to Agentic Fact-checking Systemcited by 4
31 more
  1. 2025VisRet: Visualization Improves Knowledge-Intensive Text-to-Image Retrievalcited by 3
  2. 2025Multi-Agent Retrieval-Augmented Framework for Evidence-Based Counterspeech Against Health Misinformationcited by 3
  3. 2025PASS-FC: Progressive and Adaptive Search Scheme for Fact Checking of Comprehensive Claimscited by 2
  4. 2026DECEIVE-AFC: Adversarial Claim Attacks against Search-Enabled LLM-based Fact-Checking Systemscited by 2
  5. 2026RW-Post: Auditable Evidence-Grounded Multimodal Fact-Checking in the Wildcited by 2
  6. 2025Beyond Retrieval: Improving Evidence Quality for LLM-based Multimodal Fact-Checkingcited by 1
  7. 2025MultiCheck: Strengthening Web Trust with Unified Multimodal Fact Verificationcited by 1
  8. 2026KG-ViP: Bridging Knowledge Grounding and Visual Perception in Multi-modal LLMs for Visual Question Answeringcited by 1
  9. 2026CaseFacts: A Benchmark for Legal Fact-Checking and Precedent Retrievalcited by 1
  10. 2026Fact or Fake? Assessing the Role of Deepfake Detectors in Multimodal Misinformation Detectioncited by 1
  11. 2026Multimodal Claim Extraction for Fact-Checkingcited by 1
  12. 2026When Misinformation Speaks and Converses: Rethinking Fact-Checking in Audio Platformscited by 1
  13. 2025ClaimCheck: Real-Time Fact-Checking with Small Language Modelscited by 0
  14. 2025Towards Real-Time Fake News Detection under Evidence Scarcitycited by 0
  15. 2025RW-Post: Auditable Evidence-Grounded Multimodal Fact-Checking in the Wildcited by 0
  16. 2026Multilingual, Multimodal Pipeline for Creating Authentic and Structured Fact-Checked Claim Datasetcited by 0
  17. 2026Frame-Guided Synthetic Claim Generation for Automatic Fact-Checking Using High-Volume Tabular Datacited by 0
  18. 2026Ostrakon-VL: Towards Domain-Expert MLLM for Food-Service and Retail Storescited by 0
  19. 2026Temporal Leakage in Search-Engine Date-Filtered Web Retrieval: A Retrospective Forecasting Case Studycited by 0
  20. 2026D-SECURE: Dual-Source Evidence Combination for Unified Reasoning in Misinformation Detectioncited by 0
  21. 2026Is a Picture Worth a Thousand Words? Adaptive Multimodal Fact-Checking with Visual Evidence Necessitycited by 0
  22. 2026Taxonomy of Risks on Automated Fact-Checking Systems Considering its Propagationcited by 0
  23. 2026Evidence Triangulation for Multimodal Fact-Checking in the Wildcited by 0
  24. 2026VETO: Towards Protecting Images From Frontier AI Editingcited by 0
  25. 2026How Robust Are Automated Fact-Checking Systems? A Cross-Benchmark Evaluationcited by 0
  26. 2026ElementCheck: Complexity-Aware Long-Form Text Factuality Evaluation via Sentence Elementscited by 0
  27. 2026Evaluating and Improving Evidence-Grounded Fact-Checking in LLMs via Multi-Round Evidence Ablationcited by 0
  28. 2026Self-Evolving Multimedia Verification through Memory Consolidation of Contestation Experiencescited by 0
  29. 2026MM-VeriAgent: Learning to Use Extensive Tools to Verify Multimodal Misinformation with Reinforcement Learningcited by 0
  30. 2026OpenFC: Learning Verification Policies towards Open-Search Fact Checkingcited by 0
  31. 2026EviScene: Beyond "Is It AI?" for Agentic Image Factuality Assessmentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025HintsOfTruth: A Multimodal Checkworthiness Detection Dataset with Real and Synthetic Claims
  2. 2025MuSciClaims: Multimodal Scientific Claim Verification
  3. 2024Robust Claim Verification Through Fact Detection
  4. 2025Fact in Fragments: Deconstructing Complex Claims via LLM-based Atomic Fact Extraction and Verification
  5. 2026Checking Fact with Better Retrieval: Dynamic Contrastive Learning for Evidence Retrieval
  6. 2026Novel Claim or D'ej`a Vu? Rethinking "Contamination-Free'' Dynamic Evaluation for Multimodal Automated Fact-Checking
  7. 2024Piecing It All Together: Verifying Multi-Hop Multimodal Claims
  8. 2026MIC: Explaining Image-Claim Inconsistencies in AI-Generated Multimodal Misinformation