# Interpretable Multimodal Misinformation Detection with Logic Reasoning

arXiv: https://arxiv.org/abs/2305.05964 · HTML: https://mlsanity.com/paper/2305.05964/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 56 references that are in the map.
- First posted on arXiv 10 May 2023.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Neural Probabilistic Logic Programming in DeepProbLog](https://mlsanity.com/paper/1907.08194/) (2019)
- [From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification](https://mlsanity.com/paper/1602.02068/) (2016)
- [Learning Explanatory Rules from Noisy Data](https://mlsanity.com/paper/1711.04574/) (2017)
- [Automatic Detection of Fake News](https://mlsanity.com/paper/1708.07104/) (2017)
- [Fighting Fake News: Image Splice Detection via Learned Self-Consistency](https://mlsanity.com/paper/1805.04096/) (2018)
- [Harnessing Deep Neural Networks with Logic Rules](https://mlsanity.com/paper/1603.06318/) (2016)
- [RNNLogic: Learning Logic Rules for Reasoning on Knowledge Graphs](https://mlsanity.com/paper/2010.04029/) (2020)
- [Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online Resources](https://mlsanity.com/paper/2112.00061/) (2021)
- [From Statistical Relational to Neuro-Symbolic Artificial Intelligence](https://mlsanity.com/paper/2003.08316/) (2020)
- [Learning Cross-modal Context Graph for Visual Grounding](https://mlsanity.com/paper/1911.09042/) (2019)
- [Improving Fake News Detection by Using an Entity-enhanced Framework to Fuse Diverse Multimodal Clues](https://mlsanity.com/paper/2108.10509/) (2021)
- [Inductive logic programming at 30](https://mlsanity.com/paper/2102.10556/) (2021)
- [A Survey on Multimodal Disinformation Detection](https://mlsanity.com/paper/2103.12541/) (2021)
- [Rethinking Attention-Model Explainability through Faithfulness Violation Test](https://mlsanity.com/paper/2201.12114/) (2022)
- [Generalizing to the Future: Mitigating Entity Bias in Fake News Detection](https://mlsanity.com/paper/2204.09484/) (2022)
- [Reasoning with Sarcasm by Reading In-between](https://mlsanity.com/paper/1805.02856/) (2018)
- [Is Neuro-Symbolic AI Meeting its Promise in Natural Language Processing? A Structured Review](https://mlsanity.com/paper/2202.12205/) (2022)
- [Neuro-Symbolic Inductive Logic Programming with Logical Neural Networks](https://mlsanity.com/paper/2112.03324/) (2021)
- [Memory-Guided Multi-View Multi-Domain Fake News Detection](https://mlsanity.com/paper/2206.12808/) (2022)
- [Fake News Early Detection: An Interdisciplinary Study](https://mlsanity.com/paper/1904.11679/) (2019)
- [Towards Multi-Modal Sarcasm Detection via Hierarchical Congruity Modeling with Knowledge Enhancement](https://mlsanity.com/paper/2210.03501/) (2022)
- [Bootstrapping Multi-view Representations for Fake News Detection](https://mlsanity.com/paper/2206.05741/) (2022)
- [LOREN: Logic-Regularized Reasoning for Interpretable Fact Verification](https://mlsanity.com/paper/2012.13577/) (2020)
- [AdaLoGN: Adaptive Logic Graph Network for Reasoning-Based Machine Reading Comprehension](https://mlsanity.com/paper/2203.08992/) (2022)

## Built on by (most cited first, 16 of 16)

- [DELL: Generating Reactions and Explanations for LLM-Based Misinformation Detection](https://mlsanity.com/paper/2402.10426/) (2024)
- [Evidence-Driven Retrieval Augmented Response Generation for Online Misinformation](https://mlsanity.com/paper/2403.14952/) (2024)
- [Retrieval Augmented Fact Verification by Synthesizing Contrastive Arguments](https://mlsanity.com/paper/2406.09815/) (2024)
- [CrAM: Credibility-Aware Attention Modification in LLMs for Combating Misinformation in RAG](https://mlsanity.com/paper/2406.11497/) (2024)
- [Can LLMs Improve Multimodal Fact-Checking by Asking Relevant Questions?](https://mlsanity.com/paper/2410.04616/) (2024)
- [TELLER: A Trustworthy Framework for Explainable, Generalizable and Controllable Fake News Detection](https://mlsanity.com/paper/2402.07776/) (2024)
- [Frequency Spectrum is More Effective for Multimodal Representation and Fusion: A Multimodal Spectrum Rumor Detector](https://mlsanity.com/paper/2312.11023/) (2023)
- [Exploring Cognitive and Aesthetic Causality for Multimodal Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2504.15848/) (2025)
- [A Survey on Interpretable Cross-modal Reasoning](https://mlsanity.com/paper/2309.01955/) (2023)
- [Modelling Visual Semantics via Image Captioning to extract Enhanced Multi-Level Cross-Modal Semantic Incongruity Representation with Attention for Multimodal Sarcasm Detection](https://mlsanity.com/paper/2408.02595/) (2024)
- [The Psychology of Falsehood: A Human-Centric Survey of Misinformation Detection](https://mlsanity.com/paper/2509.15896/) (2025)
- [Experiments with truth using Machine Learning: Spectral analysis and explainable classification of synthetic, false, and genuine information](https://mlsanity.com/paper/2407.05464/) (2024)
- [Dynamic Analysis and Adaptive Discriminator for Fake News Detection](https://mlsanity.com/paper/2408.10883/) (2024)
- [KGAlign: Joint Semantic-Structural Knowledge Encoding for Multimodal Fake News Detection](https://mlsanity.com/paper/2505.14714/) (2025)
- [Beyond Heuristic Prompting: A Concept-Guided Bayesian Framework for Zero-Shot Image Recognition](https://mlsanity.com/paper/2603.07911/) (2026)
- [SoK: Formal Methods for Fact-Checking and Information Integrity](https://mlsanity.com/paper/2609.23239/) (2026)

## Abstract (arXiv)

Multimodal misinformation on online social platforms is becoming a critical concern due to increasing credibility and easier dissemination brought by multimedia content, compared to traditional text-only information. While existing multimodal detection approaches have achieved high performance, the lack of interpretability hinders these systems' reliability and practical deployment. Inspired by NeuralSymbolic AI which combines the learning ability of neural networks with the explainability of symbolic learning, we propose a novel logic-based neural model for multimodal misinformation detection which integrates interpretable logic clauses to express the reasoning process of the target task. To make learning effective, we parameterize symbolic logical elements using neural representations, which facilitate the automatic generation and evaluation of meaningful logic clauses. Additionally, to make our framework generalizable across diverse misinformation sources, we introduce five meta-predicates that can be instantiated with different correlations. Results on three public datasets (Twitter, Weibo, and Sarcasm) demonstrate the feasibility and versatility of our model.