# Towards Reliable Misinformation Mitigation: Generalization, Uncertainty, and GPT-4

arXiv: https://arxiv.org/abs/2305.14928 · HTML: https://mlsanity.com/paper/2305.14928/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 63 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [Fake News Detection on Social Media: A Data Mining Perspective](https://mlsanity.com/paper/1708.01967/) (2017)
- [LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention](https://mlsanity.com/paper/2010.01057/) (2020)
- [Uncertainty Quantification and Deep Ensembles](https://mlsanity.com/paper/2007.08792/) (2020)
- [Confidence-Aware Learning for Deep Neural Networks](https://mlsanity.com/paper/2007.01458/) (2020)
- [ConvBERT: Improving BERT with Span-based Dynamic Convolution](https://mlsanity.com/paper/2008.02496/) (2020)
- [SqueezeBERT: What can computer vision teach NLP about efficient neural networks?](https://mlsanity.com/paper/2006.11316/) (2020)
- [The Surprising Performance of Simple Baselines for Misinformation Detection](https://mlsanity.com/paper/2104.06952/) (2021)
- [Adaptive Temperature Scaling for Robust Calibration of Deep Neural Networks](https://mlsanity.com/paper/2208.00461/) (2022)
- [Combating Disinformation in a Social Media Age](https://mlsanity.com/paper/2007.07388/) (2020)
- [Semi-supervised Content-based Detection of Misinformation via Tensor Embeddings](https://mlsanity.com/paper/1804.09088/) (2018)
- [On Unifying Misinformation Detection](https://mlsanity.com/paper/2104.05243/) (2021)
- [Probing Spurious Correlations in Popular Event-Based Rumor Detection Benchmarks](https://mlsanity.com/paper/2209.08799/) (2022)
- [It's All in the Embedding! Fake News Detection Using Document Embeddings](https://mlsanity.com/paper/2304.07781/) (2023)
- [Improving Generalizability of Fake News Detection Methods using Propensity Score Matching](https://mlsanity.com/paper/2002.00838/) (2020)
- [Adding Seemingly Uninformative Labels Helps in Low Data Regimes](https://mlsanity.com/paper/2008.00807/) (2020)
- [An Adversarial Benchmark for Fake News Detection Models](https://mlsanity.com/paper/2201.00912/) (2022)
- [Integrating Human-in-the-loop into Swarm Learning for Decentralized Fake News Detection](https://mlsanity.com/paper/2201.02048/) (2022)
- [AdvCat: Domain-Agnostic Robustness Assessment for Cybersecurity-Critical Applications with Categorical Inputs](https://mlsanity.com/paper/2212.13989/) (2022)
- [Is Dynamic Rumor Detection on social media Viable? An Unsupervised Perspective](https://mlsanity.com/paper/2111.11982/) (2021)

## Built on by (most cited first, 30 of 45)

- [Can LLM-Generated Misinformation Be Detected?](https://mlsanity.com/paper/2309.13788/) (2023)
- [Benchmark Data Contamination of Large Language Models: A Survey](https://mlsanity.com/paper/2406.04244/) (2024)
- [Bad Actor, Good Advisor: Exploring the Role of Large Language Models in Fake News Detection](https://mlsanity.com/paper/2309.12247/) (2023)
- [Fake News in Sheep's Clothing: Robust Fake News Detection Against LLM-Empowered Style Attacks](https://mlsanity.com/paper/2310.10830/) (2023)
- [DELL: Generating Reactions and Explanations for LLM-Based Misinformation Detection](https://mlsanity.com/paper/2402.10426/) (2024)
- [Open, Closed, or Small Language Models for Text Classification?](https://mlsanity.com/paper/2308.10092/) (2023)
- [What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detection](https://mlsanity.com/paper/2402.00371/) (2024)
- [Retrieval Augmented Fact Verification by Synthesizing Contrastive Arguments](https://mlsanity.com/paper/2406.09815/) (2024)
- [CrAM: Credibility-Aware Attention Modification in LLMs for Combating Misinformation in RAG](https://mlsanity.com/paper/2406.11497/) (2024)
- [Large Language Model Reasoning Failures](https://mlsanity.com/paper/2602.06176/) (2026)
- [Web Retrieval Agents for Evidence-Based Misinformation Detection](https://mlsanity.com/paper/2409.00009/) (2024)
- [Real-time Factuality Assessment from Adversarial Feedback](https://mlsanity.com/paper/2410.14651/) (2024)
- [PCoT: Persuasion-Augmented Chain of Thought for Detecting Fake News and Social Media Disinformation](https://mlsanity.com/paper/2506.06842/) (2025)
- [TELLER: A Trustworthy Framework for Explainable, Generalizable and Controllable Fake News Detection](https://mlsanity.com/paper/2402.07776/) (2024)
- [A Guide to Misinformation Detection Data and Evaluation](https://mlsanity.com/paper/2411.05060/) (2024)
- [Comparing GPT-4 and Open-Source Language Models in Misinformation Mitigation](https://mlsanity.com/paper/2401.06920/) (2024)
- [Improving CLIP Robustness with Knowledge Distillation and Self-Training](https://mlsanity.com/paper/2309.10361/) (2023)
- [Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Models](https://mlsanity.com/paper/2507.02870/) (2025)
- [Evaluating open-source Large Language Models for automated fact-checking](https://mlsanity.com/paper/2503.05565/) (2025)
- [On the Risk of Evidence Pollution for Malicious Social Text Detection in the Era of LLMs](https://mlsanity.com/paper/2410.12600/) (2024)
- [Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directions](https://mlsanity.com/paper/2506.11111/) (2025)
- [Supporters and Skeptics: LLM-based Analysis of Engagement with Mental Health (Mis)Information Content on Video-sharing Platforms](https://mlsanity.com/paper/2407.02662/) (2024)
- [Sword and Shield: Uses and Strategies of LLMs in Navigating Disinformation](https://mlsanity.com/paper/2506.07211/) (2025)
- [MMIDR: Teaching Large Language Model to Interpret Multimodal Misinformation via Knowledge Distillation](https://mlsanity.com/paper/2403.14171/) (2024)
- [Facts are Harder Than Opinions -- A Multilingual, Comparative Analysis of LLM-Based Fact-Checking Reliability](https://mlsanity.com/paper/2506.03655/) (2025)
- [When Scale Meets Diversity: Evaluating Language Models on Fine-Grained Multilingual Claim Verification](https://mlsanity.com/paper/2507.20700/) (2025)
- [Multi-Sourced, Multi-Agent Evidence Retrieval for Fact-Checking](https://mlsanity.com/paper/2603.00267/) (2026)
- [MALicious INTent Dataset and Inoculating LLMs for Enhanced Disinformation Detection](https://mlsanity.com/paper/2603.14525/) (2026)
- [Veracity: An Open-Source AI Fact-Checking System](https://mlsanity.com/paper/2506.15794/) (2025)
- [Defining and Evaluating Decision and Composite Risk in Language Models Applied to Natural Language Inference](https://mlsanity.com/paper/2408.01935/) (2024)

## Abstract (arXiv)

Misinformation poses a critical societal challenge, and current approaches have yet to produce an effective solution. We propose focusing on generalization, uncertainty, and how to leverage recent large language models, in order to create more practical tools to evaluate information veracity in contexts where perfect classification is impossible. We first demonstrate that GPT-4 can outperform prior methods in multiple settings and languages. Next, we explore generalization, revealing that GPT-4 and RoBERTa-large exhibit differences in failure modes. Third, we propose techniques to handle uncertainty that can detect impossible examples and strongly improve outcomes. We also discuss results on other language models, temperature, prompting, versioning, explainability, and web retrieval, each one providing practical insights and directions for future research. Finally, we publish the LIAR-New dataset with novel paired English and French misinformation data and Possibility labels that indicate if there is sufficient context for veracity evaluation. Overall, this research lays the groundwork for future tools that can drive real-world progress to combat misinformation.