mlsanityData to 14 Oct 2026

Multimodal Analytics for Real-world News using Measures of Cross-modal Entity Consistency

arXiv 2003.10421 ↗First posted 23 Mar 2020cs.CL cs.IR cs.MMcited by 33

Not on the map. Why?

Abstract (arXiv)

The World Wide Web has become a popular source for gathering information and news. Multimodal information, e.g., enriching text with photos, is typically used to convey the news more effectively or to attract attention. Photo content can range from decorative, depict additional important information, or can even contain misleading information. Therefore, automatic approaches to quantify cross-modal consistency of entity representation can support human assessors to evaluate the overall multimodal message, for instance, with regard to bias or sentiment. In some cases such measures could give hints to detect fake news, which is an increasingly important topic in today's society. In this paper, we introduce a novel task of cross-modal consistency verification in real-world news and present a multimodal approach to quantify the entity coherence between image and text. Named entity linking is applied to extract persons, locations, and events from news texts. Several measures are suggested to calculate cross-modal similarity for these entities using state of the art approaches. In contrast to previous work, our system automatically gathers example data from the Web and is applicable to real-world news. Results on two novel datasets that cover different languages, topics, and domains demonstrate the feasibility of our approach. Datasets and code are publicly available to foster research towards this new direction.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). First posted on arXiv 23 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 1; 2023: 6; 2024: 9; 2025: 8; 2026: 5.

33 papers.

Sort:

By area:

  1. 2021NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Mediacited by 107
  2. 2024SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detectioncited by 74
  3. 2023Multimodal Automated Fact-Checking: A Surveycited by 53
  4. 2021A Survey on Multimodal Disinformation Detectioncited by 47
  5. 2024DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Expertscited by 43
  6. 2023VERITE: A Robust Benchmark for Multimodal Misinformation Detection Accounting for Unimodal Biascited by 39
  7. 2023Synthetic Misinformers: Generating and Combating Multimodal Misinformationcited by 23
  8. 2023Support or Refute: Analyzing the Stance of Evidence to Detect Out-of-Context Mis- and Disinformationcited by 21
  9. 2023RED-DOT: Multimodal Fact-checking via Relevant Evidence Detectioncited by 18
  10. 2021Twitter-COMMs: Detecting Climate, COVID, and Military Multimodal Misinformationcited by 16
  11. 2023Multi-modal Fake News Detection on Social Media via Multi-grained Information Fusioncited by 15
  12. 2025COVE: COntext and VEracity prediction for out-of-context imagescited by 14
21 more
  1. 2024Multimodal Misinformation Detection by Learning from Synthetic Data with Multimodal LLMscited by 13
  2. 2024Detecting Misinformation in Multimedia Content through Cross-Modal Entity Consistency: A Dual Learning Approachcited by 12
  3. 2024Similarity over Factuality: Are we making progress on multimodal out-of-context misinformation detection?cited by 11
  4. 2025M4FC: a Multimodal, Multilingual, Multicultural, Multitask Real-World Fact-Checking Datasetcited by 7
  5. 2025CMIE: Combining MLLM Insights with External Evidence for Explainable Out-of-Context Misinformation Detectioncited by 6
  6. 2025Verifying Cross-modal Entity Consistency in News using Vision-language Modelscited by 5
  7. 2024"Image, Tell me your story!" Predicting the original meta-context of visual misinformationcited by 4
  8. 2022MONet: Multi-scale Overlap Network for Duplication Detection in Biomedical Imagescited by 3
  9. 2024Assessing News Thumbnail Representativeness: Counterfactual text can enhance the cross-modal matching abilitycited by 3
  10. 2024Image-Text Out-Of-Context Detection Using Synthetic Multimodal Misinformationcited by 2
  11. 2026The Synthetic Media Shift: Tracking the Rise, Virality, and Detectability of AI-Generated Multimodal Misinformationcited by 2
  12. 2026RW-Post: Auditable Evidence-Grounded Multimodal Fact-Checking in the Wildcited by 2
  13. 2025Mitigating GenAI-Powered Evidence Pollution for Out-Of-Context Misinformation Detectioncited by 1
  14. 2025Latent Reconstruction from Generated Data for Multimodal Misinformation Detectioncited by 1
  15. 2025Out-of-Context Misinformation Detection via Variational Domain-Invariant Learning with Test-Time Trainingcited by 1
  16. 2026Exposing Cross-Modal Consistency for Fake News Detection in Short-Form Videoscited by 1
  17. 2021A Review of Web Infodemic Analysis and Detection Trends across Multi-modalities using Deep Neural Networkscited by 0
  18. 2024SceneGraMMi: Scene Graph-boosted Hybrid-fusion for Multi-Modal Misinformation Veracity Predictioncited by 0
  19. 2025RW-Post: Auditable Evidence-Grounded Multimodal Fact-Checking in the Wildcited by 0
  20. 2026ReMMD: Realistic Multilingual Multi-Image Agentic Verification for Multimodal Misinformation Detectioncited by 0
  21. 2026Evidence Triangulation for Multimodal Fact-Checking in the Wildcited by 0