mlsanityData to 14 Oct 2026

Visual News: Benchmark and Challenges in News Image Captioning

arXiv 2010.03743 ↗First posted 8 Oct 2020cs.CVcited by 121

Not on the map. Why?

Cited by 27 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose Visual News Captioner, an entity-aware model for the task of news image captioning. We also introduce Visual News, a large-scale benchmark consisting of more than one million news images along with associated news articles, image captions, author information, and other metadata. Unlike the standard image captioning task, news images depict situations where people, locations, and events are of paramount importance. Our proposed method can effectively combine visual and textual features to generate captions with richer information such as events and entities. More specifically, built upon the Transformer architecture, our model is further equipped with novel multi-modal feature fusion techniques and attention mechanisms, which are designed to generate named entities more accurately. Our method utilizes much fewer parameters while achieving slightly better prediction results than competing methods. Our larger and more diverse Visual News dataset further highlights the remaining challenges in captioning news images.

Key facts. Cited by 121 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 5; 2023: 28; 2024: 28; 2025: 31; 2026: 26.

121 papers.

Sort:

By area:

  1. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
  2. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 445
  3. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 228
  4. 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMscited by 170
  5. 2023UniIR: Training and Benchmarking Universal Multimodal Information Retrieverscited by 166
  6. 2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMscited by 161
  7. 2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 138
  8. 2023Open-domain Visual Entity Recognition: Towards Recognizing Millions of Wikipedia Entitiescited by 113
  9. 2023Detecting and Grounding Multi-Modal Media Manipulationcited by 99
  10. 2023Evaluation and Analysis of Hallucination in Large Vision-Language Modelscited by 99
  11. 2024Detecting Multimedia Generated by Large AI Models: A Surveycited by 76
  12. 2024SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detectioncited by 74
38 more
  1. 2021Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online Resourcescited by 71
  2. 2024MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMscited by 61
  3. 2023Detecting and Grounding Multi-Modal Media Manipulation and Beyondcited by 56
  4. 2023Do LVLMs Understand Charts? Analyzing and Correcting Factual Errors in Chart Captioningcited by 49
  5. 2023AutoSplice: A Text-prompt Manipulated Image Dataset for Media Forensicscited by 46
  6. 2023VERITE: A Robust Benchmark for Multimodal Misinformation Detection Accounting for Unimodal Biascited by 39
  7. 2025MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interactioncited by 39
  8. 2024FKA-Owl: Advancing Multimodal Fake News Detection through Knowledge-Augmented LVLMscited by 38
  9. 2023Towards Understanding In-Context Learning with Contrastive Demonstrations and Saliency Mapscited by 35
  10. 2022WikiDiverse: A Multimodal Entity Linking Dataset with Diversified Contextual Topics and Entity Typescited by 34
  11. 2022Multimodal Knowledge Alignment with Reinforcement Learningcited by 30
  12. 2023EDIS: Entity-Driven Image Search over Multimodal Web Contentcited by 27
  13. 2022I Can't Believe There's No Images! Learning Visual Tasks Using only Language Supervisioncited by 26
  14. 2024MM-Soc: Benchmarking Multimodal Large Language Models in Social Media Platformscited by 26
  15. 2025Modality Curation: Building Universal Embeddings for Advanced Multimodal Information Retrievalcited by 26
  16. 2023Synthetic Misinformers: Generating and Combating Multimodal Misinformationcited by 23
  17. 2025Improve Multi-Modal Embedding Learning via Explicit Hard Negative Gradient Amplifyingcited by 23
  18. 2023Rethinking Image Editing Detection in the Era of Generative AI Revolutioncited by 21
  19. 2024Can LLMs Improve Multimodal Fact-Checking by Asking Relevant Questions?cited by 17
  20. 2023COVID-VTS: Fact Extraction and Verification on Short Video Platformscited by 15
  21. 2024MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language Modelscited by 15
  22. 2025MIEB: Massive Image Embedding Benchmarkcited by 15
  23. 2023Unified Frequency-Assisted Transformer Framework for Detecting and Grounding Multi-Modal Manipulationcited by 14
  24. 2025COVE: COntext and VEracity prediction for out-of-context imagescited by 14
  25. 2024Multimodal Misinformation Detection by Learning from Synthetic Data with Multimodal LLMscited by 13
  26. 2023Visually-Aware Context Modeling for News Image Captioningcited by 12
  27. 2024TIGeR: Unifying Text-to-Image Generation and Retrieval with Large Multimodal Modelscited by 12
  28. 2025Closing the Modality Gap for Mixed Modality Searchcited by 12
  29. 2024Similarity over Factuality: Are we making progress on multimodal out-of-context misinformation detection?cited by 11
  30. 2022Show, Interpret and Tell: Entity-aware Contextualised Image Captioning in Wikipediacited by 10
  31. 2025U-MARVEL: Unveiling Key Factors for Universal Multimodal Retrieval via Embedding Learning with MLLMscited by 10
  32. 2024XL-HeadTags: Leveraging Multimodal Retrieval Augmentation for the Multilingual Generation of News Headlines and Tagscited by 9
  33. 2023Interpretable Detection of Out-of-Context Misinformation with Neural-Symbolic-Enhanced Large Multimodal Modelcited by 8
  34. 2024Credible, Unreliable or Leaked?: Evidence Verification for Enhanced Automated Fact-checkingcited by 8
  35. 2025The Coherence Trap: When MLLM-Crafted Narratives Exploit Manipulated Visual Contextscited by 8
  36. 2025RadarQA: Multi-modal Quality Analysis of Weather Radar Forecastscited by 8
  37. 2023"Let's not Quote out of Context": Unified Vision-Language Pretraining for Context Assisted Image Captioningcited by 7
  38. 2023DocumentCLIP: Linking Figures and Main Body Text in Reflowed Documentscited by 7

50 most cited of 121 All 121 (JSON)