# Combating Misinformation in the Age of LLMs: Opportunities and Challenges

arXiv: https://arxiv.org/abs/2311.05656 · HTML: https://mlsanity.com/paper/2311.05656/

## Key facts

- Cited by 95 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Nov 2023.

## Built on by (most cited first, 30 of 95)

- [A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2312.02003/) (2023)
- [OASIS: Open Agent Social Interaction Simulations with One Million Agents](https://mlsanity.com/paper/2411.11581/) (2024)
- [A Survey of Text Watermarking in the Era of Large Language Models](https://mlsanity.com/paper/2312.07913/) (2023)
- [Jatmo: Prompt Injection Defense by Task-Specific Finetuning](https://mlsanity.com/paper/2312.17673/) (2023)
- [Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing](https://mlsanity.com/paper/2402.16192/) (2024)
- [Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents](https://mlsanity.com/paper/2505.02077/) (2025)
- [Introducing v0.5 of the AI Safety Benchmark from MLCommons](https://mlsanity.com/paper/2404.12241/) (2024)
- [Boosting Jailbreak Attack with Momentum](https://mlsanity.com/paper/2405.01229/) (2024)
- [Decoding by Contrasting Knowledge: Enhancing LLMs' Confidence on Edited Facts](https://mlsanity.com/paper/2405.11613/) (2024)
- [AI and the Problem of Knowledge Collapse](https://mlsanity.com/paper/2404.03502/) (2024)
- [Persuasion with Large Language Models: A Survey of Empirical Evidence, Study Methodologies, and Ethical Implications](https://mlsanity.com/paper/2411.06837/) (2024)
- [Exploiting Novel GPT-4 APIs](https://mlsanity.com/paper/2312.14302/) (2023)
- [Navigating LLM Ethics: Advancements, Challenges, and Future Directions](https://mlsanity.com/paper/2406.18841/) (2024)
- [Can Knowledge Editing Really Correct Hallucinations?](https://mlsanity.com/paper/2410.16251/) (2024)
- [LLM Can be a Dangerous Persuader: Empirical Study of Persuasion Safety in Large Language Models](https://mlsanity.com/paper/2504.10430/) (2025)
- [Can Editing LLMs Inject Harm?](https://mlsanity.com/paper/2407.20224/) (2024)
- [Double-I Watermark: Protecting Model Copyright for LLM Fine-tuning](https://mlsanity.com/paper/2402.14883/) (2024)
- [Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Models](https://mlsanity.com/paper/2405.15349/) (2024)
- [Mark Your LLM: Detecting the Misuse of Open-Source Large Language Models via Watermarking](https://mlsanity.com/paper/2503.04636/) (2025)
- [LLM-Generated Fake News Induces Truth Decay in News Ecosystem: A Case Study on Neural News Recommendation](https://mlsanity.com/paper/2504.20013/) (2025)
- [Knowledge Editing in Language Models via Adapted Direct Preference Optimization](https://mlsanity.com/paper/2406.09920/) (2024)
- [Analysing the Residual Stream of Language Models Under Knowledge Conflicts](https://mlsanity.com/paper/2410.16090/) (2024)
- [Optimizing Adaptive Attacks against Watermarks for Language Models](https://mlsanity.com/paper/2410.02440/) (2024)
- [Research Trends for the Interplay between Large Language Models and Knowledge Graphs](https://mlsanity.com/paper/2406.08223/) (2024)
- [Position: Editing Large Language Models Poses Serious Safety Risks](https://mlsanity.com/paper/2502.02958/) (2025)
- [Exploring the Robustness of In-Context Learning with Noisy Labels](https://mlsanity.com/paper/2404.18191/) (2024)
- [Web Retrieval Agents for Evidence-Based Misinformation Detection](https://mlsanity.com/paper/2409.00009/) (2024)
- [BenchmarkCards: Standardized Documentation for Large Language Model Benchmarks](https://mlsanity.com/paper/2410.12974/) (2024)
- [A Guide to Misinformation Detection Data and Evaluation](https://mlsanity.com/paper/2411.05060/) (2024)
- [Large Language Models are Vulnerable to Bait-and-Switch Attacks for Generating Harmful Content](https://mlsanity.com/paper/2402.13926/) (2024)

## Abstract (arXiv)

Misinformation such as fake news and rumors is a serious threat on information ecosystems and public trust. The emergence of Large Language Models (LLMs) has great potential to reshape the landscape of combating misinformation. Generally, LLMs can be a double-edged sword in the fight. On the one hand, LLMs bring promising opportunities for combating misinformation due to their profound world knowledge and strong reasoning abilities. Thus, one emergent question is: how to utilize LLMs to combat misinformation? On the other hand, the critical challenge is that LLMs can be easily leveraged to generate deceptive misinformation at scale. Then, another important question is: how to combat LLM-generated misinformation? In this paper, we first systematically review the history of combating misinformation before the advent of LLMs. Then we illustrate the current efforts and present an outlook for these two fundamental questions respectively. The goal of this survey paper is to facilitate the progress of utilizing LLMs for fighting misinformation and call for interdisciplinary efforts from different stakeholders for combating LLM-generated misinformation.