mlsanityData to 13 Oct 2026

COVE: COntext and VEracity prediction for out-of-context images

arXiv 2502.01194 ↗First posted 3 Feb 2025cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

Images taken out of their context are the most prevalent form of multimodal misinformation. Debunking them requires (1) providing the true context of the image and (2) checking the veracity of the image's caption. However, existing automated fact-checking methods fail to tackle both objectives explicitly. In this work, we introduce COVE, a new method that predicts first the true COntext of the image and then uses it to predict the VEracity of the caption. COVE beats the SOTA context prediction model on all context items, often by more than five percentage points. It is competitive with the best veracity prediction models on synthetic data and outperforms them on real-world data, showing that it is beneficial to combine the two tasks sequentially. Finally, we conduct a human study that reveals that the predicted context is a reusable and interpretable artifact to verify new out-of-context captions for the same image. Our code and data are made available.

Key facts. Cited by 14 papers in the map (data through 13 Oct 2026). Builds on 25 of its 49 references that are in the map. First posted on arXiv 3 Feb 2025.

Builds on

25 of 49 references.

References · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,831
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,480
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,389
  5. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,068
  6. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,864
  7. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,239
  8. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,169
  9. 2020Autoregressive Entity Retrievalcited by 333
  10. 2023Fact-Checking Complex Claims with Program-Guided Reasoningcited by 146
  11. 2023AVeriTeC: A Dataset for Real-world Claim Verification with Evidence from the Webcited by 144
  12. 2020Visual News: Benchmark and Challenges in News Image Captioningcited by 121
13 more
  1. 2023Open-domain Visual Entity Recognition: Towards Recognizing Millions of Wikipedia Entitiescited by 112
  2. 2021NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Mediacited by 106
  3. 2024SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detectioncited by 73
  4. 2021Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online Resourcescited by 71
  5. 2023WikiChat: Stopping the Hallucination of Large Language Model Chatbots by Few-Shot Grounding on Wikipediacited by 67
  6. 2024DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Expertscited by 42
  7. 2020Multimodal Analytics for Real-world News using Measures of Cross-modal Entity Consistencycited by 33
  8. 2018Deep Multimodal Image-Repurposing Detectioncited by 31
  9. 2024AMMeBa: A Large-Scale Survey and Dataset of Media-Based Misinformation In-The-Wildcited by 27
  10. 2023QACHECK: A Demonstration System for Question-Guided Multi-Hop Fact-Checkingcited by 26
  11. 2023Support or Refute: Analyzing the Stance of Evidence to Detect Out-of-Context Mis- and Disinformationcited by 21
  12. 2024Generating Media Background Checks for Automated Source Critical Reasoningcited by 10
  13. 2024Credible, Unreliable or Leaked?: Evidence Verification for Enhanced Automated Fact-checkingcited by 8

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 4; 2026: 10.

14 papers.

Sort:

  1. 2025M4FC: a Multimodal, Multilingual, Multicultural, Multitask Real-World Fact-Checking Datasetcited by 7
  2. 2026VeriTaS: The First Dynamic Benchmark for Multimodal Automated Fact-Checkingcited by 5
  3. 2025T^2Agent A Tool-augmented Multimodal Misinformation Detection Agent with Monte Carlo Tree Searchcited by 4
  4. 2025GETReason: Enhancing Image Context Extraction through Hierarchical Multi-Agent Reasoningcited by 3
  5. 2025Evidence-Grounded Multimodal Misinformation Detection with Attention-Based GNNscited by 2
  6. 2026The Synthetic Media Shift: Tracking the Rise, Virality, and Detectability of AI-Generated Multimodal Misinformationcited by 2
  7. 2026ChartAttack: Testing the Vulnerability of LLMs to Malicious Prompting in Chart Generationcited by 0
  8. 2026NewsRECON: News Article Retrieval for Image Contextualizationcited by 0
  9. 2026Knowledge-Intensive Video Generationcited by 0
  10. 2026The Warrant Gap: Claim-Conditioned Re-scoring for Fact-Checkingcited by 0
  11. 2026NepOOC-M: Bilingual Nepali-English Benchmark and Comparative Analysis of Multimodal Architectures for OOC Detectioncited by 0
  12. 2026RCMN: Understanding Misleadingness in Influential Public Discoursecited by 0
2 more
  1. 2026UNMATCH: Selective Unbalanced Token-Patch Matching for Forensic Image-Claim Verificationcited by 0
  2. 2026MIC: Explaining Image-Claim Inconsistencies in AI-Generated Multimodal Misinformationcited by 0