mlsanityData to 14 Oct 2026

The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation

arXiv 2312.09085 ↗First posted 14 Dec 2023cs.CL cs.AI cs.CRcited by 109

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) encapsulate vast amounts of knowledge but still remain vulnerable to external misinformation. Existing research mainly studied this susceptibility behavior in a single-turn setting. However, belief can change during a multi-turn conversation, especially a persuasive one. Therefore, in this study, we delve into LLMs' susceptibility to persuasive conversations, particularly on factual questions that they can answer correctly. We first curate the Farm (i.e., Fact to Misinform) dataset, which contains factual questions paired with systematically generated persuasive misinformation. Then, we develop a testing framework to track LLMs' belief changes in a persuasive dialogue. Through extensive experiments, we find that LLMs' correct beliefs on factual knowledge can be easily manipulated by various persuasive strategies.

Key facts. Cited by 109 papers in the map (data through 14 Oct 2026). Builds on 41 of its 97 references that are in the map. First posted on arXiv 14 Dec 2023.

Builds on

41 of 97 references.

References · 41

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  8. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  9. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  10. 2019Language Models as Knowledge Bases?cited by 1,936
  11. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  12. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
29 more
  1. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  2. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  3. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  4. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  5. 2019How Can We Know What Language Models Know?cited by 959
  6. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  7. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 801
  8. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  9. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 740
  10. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  11. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  12. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 571
  13. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 377
  14. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 341
  15. 2023Are aligned neural networks adversarially aligned?cited by 315
  16. 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
  17. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 243
  18. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 206
  19. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 202
  20. 2023Can LLM-Generated Misinformation Be Detected?cited by 177
  21. 2022Large Language Models with Controllable Working Memorycited by 169
  22. 2019oLMpics -- On what Language Model Pre-training Capturescited by 161
  23. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  24. 2023Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentationcited by 147
  25. 2022Parallel Context Windows for Large Language Modelscited by 81
  26. 2023AI model GPT-3 (dis)informs us better than humanscited by 78
  27. 2023Disinformation Detection: An Evolving Challenge in the Age of LLMscited by 41
  28. 2023Dialectical language model evaluation: An initial appraisal of the commonsense spatial reasoning abilities of LLMscited by 20
  29. 2022Social Influence Dialogue Systems: A Survey of Datasets and Models For Social Influence Taskscited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 29; 2025: 44; 2026: 36.

109 papers.

Sort:

By area:

  1. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 573
  2. 2024Knowledge Conflicts for LLMs: A Surveycited by 283
  3. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 121
  4. 2025Retrieval-Augmented Generation with Conflicting Evidencecited by 63
  5. 2024A Safe Harbor for AI Evaluation and Red Teamingcited by 47
  6. 2024Understanding the Dark Side of LLMs' Intrinsic Self-Correctioncited by 42
  7. 2024DELL: Generating Reactions and Explanations for LLM-Based Misinformation Detectioncited by 41
  8. 2024Knowledge Boundary of Large Language Models: A Surveycited by 40
  9. 2024DYNAMICQA: Tracing Internal Knowledge Conflicts in Language Modelscited by 34
  10. 2025MMRC: A Large-Scale Benchmark for Understanding Multimodal Large Language Model in Real-World Conversationcited by 29
  11. 2024Navigating LLM Ethics: Advancements, Challenges, and Future Directionscited by 28
  12. 2025Nuclear Deployed: Analyzing Catastrophic Risks in Decision-making of Autonomous LLM Agentscited by 27
38 more
  1. 2025Trust Me, I'm Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answercited by 27
  2. 2025Persuade Me if You Can: A Framework for Evaluating Persuasion Effectiveness and Susceptibility Among Large Language Modelscited by 27
  3. 2024What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detectioncited by 25
  4. 2024"They are uncultured": Unveiling Covert Harms and Social Threats in LLM Generated Conversationscited by 25
  5. 2024Fundamental Problems With Model Editing: How Should Rational Belief Revision Work in LLMs?cited by 24
  6. 2024A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarioscited by 23
  7. 2025LLM Can be a Dangerous Persuader: Empirical Study of Persuasion Safety in Large Language Modelscited by 23
  8. 2026Sycophancy Hides Linearly in the Attention Headscited by 23
  9. 2025ReLearn: Unlearning via Learning for Large Language Modelscited by 22
  10. 2024Language Models in Dialogue: Conversational Maxims for Human-AI Interactionscited by 21
  11. 2024LLM-Generated Black-box Explanations Can Be Adversarially Helpfulcited by 21
  12. 2024Constructing Benchmarks and Interventions for Combating Hallucinations in LLMscited by 19
  13. 2024Preemptive Answer "Attacks" on Chain-of-Thought Reasoningcited by 19
  14. 2024Teaching Models to Balance Resisting and Accepting Persuasioncited by 19
  15. 2026Lying with Truths: Open-Channel Multi-Agent Collusion for Belief Manipulation via Generative Montagecited by 19
  16. 2025How to Protect Yourself from 5G Radiation? Investigating LLM Responses to Implicit Misinformationcited by 18
  17. 2024Let Silence Speak: Enhancing Fake News Detection with Generated Comments from Large Language Modelscited by 17
  18. 2025PersuasiveToM: A Benchmark for Evaluating Machine Theory of Mind in Persuasive Dialoguescited by 17
  19. 2024Distinguishing Ignorance from Error in LLM Hallucinationscited by 14
  20. 2025Epistemic Alignment: A Mediating Framework for User-LLM Knowledge Deliverycited by 14
  21. 2025Firm or Fickle? Evaluating Large Language Models Consistency in Sequential Interactionscited by 13
  22. 2025Must Read: A Comprehensive Survey of Computational Persuasioncited by 13
  23. 2025Persuasion Dynamics in LLMs: Investigating Robustness and Adaptability in Knowledge and Safety with DuET-PDcited by 13
  24. 2025When Persuasion Overrides Truth in Multi-Agent LLM Debates: Introducing a Confidence-Weighted Persuasion Override Rate (CW-POR)cited by 12
  25. 2024FB-Bench: A Fine-Grained Multi-Task Benchmark for Evaluating LLMs' Responsiveness to Human Feedbackcited by 11
  26. 2025Feedback Friction: LLMs Struggle to Fully Incorporate External Feedbackcited by 11
  27. 2024EgoSocialArena: Benchmarking the Social Intelligence of Large Language Models from a First-person Perspectivecited by 10
  28. 2024Moral Persuasion in Large Language Models: Evaluating Susceptibility and Ethical Alignmentcited by 10
  29. 2025Exploring Multimodal Challenges in Toxic Chinese Detection: Taxonomy, Benchmark, and Findingscited by 10
  30. 2025LLM Sensitivity Evaluation Framework for Clinical Diagnosiscited by 7
  31. 2025AI Awarenesscited by 7
  32. 2025Helpful Agent Meets Deceptive Judge: Understanding Vulnerabilities in Agentic Workflowscited by 7
  33. 2025Stop Reducing Responsibility in LLM-Powered Multi-Agent Systems to Local Alignmentcited by 7
  34. 2026Do Images Speak Louder than Words? Investigating the Effect of Textual Misinformation in VLMscited by 7
  35. 2025Aligning Large Language Models for Faithful Integrity Against Opposing Argumentcited by 6
  36. 2025HACK: Hallucinations Along Certainty and Knowledge Axescited by 6
  37. 2026Seeing through the Conflict: Transparent Knowledge Conflict Handling in Retrieval-Augmented Generationcited by 6
  38. 2026When Correct Beliefs Collapse: Epistemic Resilience of LLMs under Clinical Pressurecited by 6

50 most cited of 109 All 109 (JSON)