mlsanityData to 7 Oct 2026

LEMMA: Towards LVLM-Enhanced Multimodal Misinformation Detection with External Knowledge Augmentation

arXiv 2402.11943 ↗First posted 19 Feb 2024cs.CLcited by 28

Not on the map. Why?

Abstract (arXiv)

The rise of multimodal misinformation on social platforms poses significant challenges for individuals and societies. Its increased credibility and broader impact compared to textual misinformation make detection complex, requiring robust reasoning across diverse media types and profound knowledge for accurate verification. The emergence of Large Vision Language Model (LVLM) offers a potential solution to this problem. Leveraging their proficiency in processing visual and textual information, LVLM demonstrates promising capabilities in recognizing complex information and exhibiting strong reasoning skills. In this paper, we first investigate the potential of LVLM on multimodal misinformation detection. We find that even though LVLM has a superior performance compared to LLMs, its profound reasoning may present limited power with a lack of evidence. Based on these observations, we propose LEMMA: LVLM-Enhanced Multimodal Misinformation Detection with External Knowledge Augmentation. LEMMA leverages LVLM intuition and reasoning capabilities while augmenting them with external knowledge to enhance the accuracy of misinformation detection. Our method improves the accuracy over the top baseline LVLM by 7% and 13% on Twitter and Fakeddit datasets respectively.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 15 of its 40 references that are in the map. First posted on arXiv 19 Feb 2024.

Builds on

15 of 40 references.

References · 15

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  3. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  4. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  5. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  6. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  7. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  8. 2023FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarioscited by 247
  9. 2023LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agentscited by 188
  10. 2023Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkerscited by 86
  11. 2019r/Fakeddit: A New Multimodal Benchmark Dataset for Fine-grained Fake News Detectioncited by 53
  12. 2018On the Origins of Memes by Means of Fringe Web Communitiescited by 45
3 more
  1. 2024RAG-Fusion: a New Take on Retrieval-Augmented Generationcited by 42
  2. 2023FLEEK: Factual Error Detection and Correction with Evidence Retrieved from External Knowledgecited by 24
  3. 2021FNR: A Similarity and Transformer-Based Approach to Detect Multi-Modal Fake News in Social Mediacited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 14; 2026: 7.

28 papers.

Sort:

By area:

  1. 2024MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Modelscited by 41
  2. 2024Let Silence Speak: Enhancing Fake News Detection with Generated Comments from Large Language Modelscited by 16
  3. 2024RAEmoLLM: Retrieval Augmented LLMs for Cross-Domain Misinformation Detection Using In-Context Learning Based on Emotional Informationcited by 14
  4. 2024Multimodal Misinformation Detection by Learning from Synthetic Data with Multimodal LLMscited by 13
  5. 2024GAMED: Knowledge Adaptive Multi-Experts Decoupling for Multimodal Fake News Detectioncited by 11
  6. 2025TRUST-VL: An Explainable News Assistant for General Multimodal Misinformation Detectioncited by 10
  7. 2025CMIE: Combining MLLM Insights with External Evidence for Explainable Out-of-Context Misinformation Detectioncited by 6
  8. 2025XFacta: Contemporary, Real-World Dataset and Evaluation for Multimodal Misinformation Detection with Multimodal LLMscited by 6
  9. 2026The Paradigm Shift: A Comprehensive Survey on Large Vision Language Models for Multimodal Fake News Detectioncited by 6
  10. 2025Seeing Through Deception: Uncovering Misleading Creator Intent in Multimodal News with Vision-Language Modelscited by 5
  11. 2025A Survey on Proactive Defense Strategies Against Misinformation in Large Language Modelscited by 5
  12. 2025Can Large Vision-Language Models Understand Multimodal Sarcasm?cited by 5
16 more
  1. 2024Fake News Detection: Comparative Evaluation of BERT-like Models and Large Language Models with Generative AI-Annotated Datacited by 4
  2. 2026What's Left Unsaid? Detecting and Correcting Misleading Omissions in Multimodal News Previewscited by 3
  3. 2025Evidence-Grounded Multimodal Misinformation Detection with Attention-Based GNNscited by 2
  4. 2025E-CaTCH: Event-Centric Cross-Modal Attention with Temporal Consistency and Class-Imbalance Handling for Misinformation Detectioncited by 1
  5. 2025Effectiveness of Large Multimodal Models in Detecting Disinformation: Experimental Resultscited by 1
  6. 2025Enhancing Multimodal Misinformation Detection by Replaying the Whole Story from Image Modality Perspectivecited by 1
  7. 2026They Said Memes Were Harmless-We Found the Ones That Hurt: Decoding Jokes, Symbols, and Cultural Referencescited by 1
  8. 2026Benchmarking Deflection and Hallucination in Large Vision-Language Modelscited by 1
  9. 2026RW-Post: Auditable Evidence-Grounded Multimodal Fact-Checking in the Wildcited by 1
  10. 2024Dynamic Analysis and Adaptive Discriminator for Fake News Detectioncited by 0
  11. 2025Beyond Retrieval: Improving Evidence Quality for LLM-based Multimodal Fact-Checkingcited by 0
  12. 2025Augmented Vision-Language Models: A Systematic Reviewcited by 0
  13. 2025MERIT: Modular Framework for Multimodal Misinformation Detection with Web-Grounded Reasoningcited by 0
  14. 2025RW-Post: Auditable Evidence-Grounded Multimodal Fact-Checking in the Wildcited by 0
  15. 2026Multimodal Climate Disinformation Detection: Integrating Vision-Language Models with External Knowledge Sourcescited by 0
  16. 2026MOMENTA: Mixture-of-Experts Over Multimodal Embeddings with Neural Temporal Aggregation for Misinformation Detectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025E2LVLM:Evidence-Enhanced Large Vision-Language Model for Multimodal Out-of-Context Misinformation Detection
  2. 2024Multimodal Misinformation Detection using Large Vision-Language Models
  3. 2024MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language Models
  4. 2024Large Visual-Language Models Are Also Good Classifiers: A Study of In-Context Multimodal Fake News Detection
  5. 2025MMD-Thinker: Adaptive Multi-Dimensional Thinking for Multimodal Misinformation Detection
  6. 2024SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detection
  7. 2025DiFaR: Enhancing Multimodal Misinformation Detection with Diverse, Factual, and Relevant Rationales
  8. 2024MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMs