mlsanityData to 14 Oct 2026

Can LLM-Generated Misinformation Be Detected?

arXiv 2309.13788 ↗First posted 25 Sep 2023cs.CL cs.AI cs.CRcited by 177

Not on the map. Why?

Cited by 28 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The advent of Large Language Models (LLMs) has made a transformative impact. However, the potential that LLMs such as ChatGPT can be exploited to generate misinformation has posed a serious concern to online safety and public trust. A fundamental research question is: will LLM-generated misinformation cause more harm than human-written misinformation? We propose to tackle this question from the perspective of detection difficulty. We first build a taxonomy of LLM-generated misinformation. Then we categorize and validate the potential real-world methods for generating misinformation with LLMs. Then, through extensive empirical investigation, we discover that LLM-generated misinformation can be harder to detect for humans and detectors compared to human-written misinformation with the same semantics, which suggests it can have more deceptive styles and potentially cause more harm. We also discuss the implications of our discovery on combating misinformation in the age of LLMs and the countermeasures.

Key facts. Cited by 177 papers in the map (data through 14 Oct 2026). Builds on 231 of its 362 references that are in the map. First posted on arXiv 25 Sep 2023.

Builds on

231 of 362 references.

References · 231

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  8. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  10. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  11. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  12. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
38 more
  1. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  2. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  3. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  4. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  5. 2023A Survey of Large Language Modelscited by 2,576
  6. 2022Emergent Abilities of Large Language Modelscited by 2,512
  7. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  8. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  9. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  10. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  11. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  12. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  13. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  14. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  15. 2023LIMA: Less Is More for Alignmentcited by 1,100
  16. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  17. 2022Red Teaming Language Models with Language Modelscited by 1,026
  18. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  19. 2021Ethical and social risks of harm from Language Modelscited by 966
  20. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  21. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  22. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
  23. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
  24. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  25. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  26. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
  27. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 726
  28. 2023A Watermark for Large Language Modelscited by 725
  29. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  30. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  31. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  32. 2023A Comprehensive Overview of Large Language Modelscited by 648
  33. 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
  34. 2022Towards Reasoning in Large Language Models: A Surveycited by 596
  35. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
  36. 2019Defending Against Neural Fake Newscited by 559
  37. 2022Improving alignment of dialogue agents via targeted human judgementscited by 531
  38. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 518

50 most cited of 231 All 231 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 12; 2024: 82; 2025: 56; 2026: 27.

177 papers.

Sort:

By area:

  1. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
  2. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 387
  3. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 283
  4. 2024Knowledge Conflicts for LLMs: A Surveycited by 283
  5. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 240
  6. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 186
  7. 2023A Semantic Invariant Robust Watermark for Large Language Modelscited by 112
  8. 2023A Survey of Text Watermarking in the Era of Large Language Modelscited by 112
  9. 2023The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversationcited by 109
  10. 2024Detecting Multimedia Generated by Large AI Models: A Surveycited by 76
  11. 2023Mark My Words: Analyzing and Evaluating Language Model Watermarkscited by 71
  12. 2023Fake News in Sheep's Clothing: Robust Fake News Detection Against LLM-Empowered Style Attackscited by 70
38 more
  1. 2024Building Guardrails for Large Language Modelscited by 65
  2. 2024Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineeringcited by 63
  3. 2024Introducing v0.5 of the AI Safety Benchmark from MLCommonscited by 62
  4. 2024MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMscited by 61
  5. 2025From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agentscited by 51
  6. 2024Disentangling Memory and Reasoning Ability in Large Language Modelscited by 44
  7. 2024DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Expertscited by 43
  8. 2024Understanding the Dark Side of LLMs' Intrinsic Self-Correctioncited by 42
  9. 2024DELL: Generating Reactions and Explanations for LLM-Based Misinformation Detectioncited by 41
  10. 2024Can Large Language Models Identify Authorship?cited by 41
  11. 2023Neural Retrievers are Biased Towards LLM-Generated Contentcited by 38
  12. 2025GUARDIAN: Safeguarding LLM Multi-Agent Collaborations with Temporal Graph Modelingcited by 36
  13. 2024Can Watermarks Survive Translation? On the Cross-lingual Consistency of Text Watermark for Large Language Modelscited by 33
  14. 2024How Susceptible are LLMs to Influence in Prompts?cited by 33
  15. 2024Evidence-Driven Retrieval Augmented Response Generation for Online Misinformationcited by 32
  16. 2024Exploring the Deceptive Power of LLM-Generated Fake News: A Study of Real-World Detection Challengescited by 30
  17. 2024Scaling Trends for Data Poisoning in LLMscited by 30
  18. 2023Exploiting Novel GPT-4 APIscited by 28
  19. 2024Bypassing LLM Watermarks with Color-Aware Substitutionscited by 28
  20. 2024Navigating LLM Ethics: Advancements, Challenges, and Future Directionscited by 28
  21. 2024Can Knowledge Editing Really Correct Hallucinations?cited by 28
  22. 2024HaluEval-Wild: Evaluating Hallucinations of Language Models in the Wildcited by 27
  23. 2024From Sora What We Can See: A Survey of Text-to-Video Generationcited by 27
  24. 2024Securing Large Language Models: Addressing Bias, Misinformation, and Prompt Attackscited by 26
  25. 2023FFT: Towards Harmlessness Evaluation and Analysis for LLMs with Factuality, Fairness, Toxicitycited by 25
  26. 2024What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detectioncited by 25
  27. 2024TELEClass: Taxonomy Enrichment and LLM-Enhanced Hierarchical Text Classification with Minimal Supervisioncited by 25
  28. 2024Retrieval Augmented Fact Verification by Synthesizing Contrastive Argumentscited by 25
  29. 2024Under the Surface: Tracking the Artifactuality of LLM-Generated Datacited by 24
  30. 2024Detecting AI-Generated Text: Factors Influencing Detectability with Current Methodscited by 23
  31. 2024Preference Tuning with Human Feedback on Language, Speech, and Vision Tasks: A Surveycited by 23
  32. 2024CrAM: Credibility-Aware Attention Modification in LLMs for Combating Misinformation in RAGcited by 22
  33. 2024Can Editing LLMs Inject Harm?cited by 22
  34. 2024Not All Contexts Are Equal: Teaching LLMs Credibility-aware Generationcited by 21
  35. 2024Depending on yourself when you should: Mentoring LLM with RL agents to become the master in cybersecurity gamescited by 20
  36. 2024Authorship Obfuscation in Multilingual Machine-Generated Text Detectioncited by 19
  37. 2024A Survey of AI-generated Text Forensic Systems: Detection, Attribution, and Characterizationcited by 19
  38. 2024Generative Large Language Models in Automated Fact-Checking: A Surveycited by 19

50 most cited of 177 All 177 (JSON)