mlsanityData to 7 Oct 2026

Lies, Damned Lies, and Distributional Language Statistics: Persuasion and Deception with Large Language Models

arXiv 2412.17128 ↗First posted 22 Dec 2024cs.CL cs.CY cs.HCcited by 14

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) can generate content that is as persuasive as human-written text and appear capable of selectively producing deceptive outputs. These capabilities raise concerns about potential misuse and unintended consequences as these systems become more widely deployed. This review synthesizes recent empirical work examining LLMs' capacity and proclivity for persuasion and deception, analyzes theoretical risks that could arise from these capabilities, and evaluates proposed mitigations. While current persuasive effects are relatively small, various mechanisms could increase their impact, including fine-tuning, multimodality, and social factors. We outline key open questions for future research, including how persuasive AI systems might become, whether truth enjoys an inherent advantage over falsehoods, and how effective different mitigation strategies may be in practice.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 75 of its 266 references that are in the map. First posted on arXiv 22 Dec 2024.

Builds on

75 of 266 references.

References · 75

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2020Scaling Laws for Neural Language Modelscited by 6,411
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  4. 2017Deep reinforcement learning from human preferencescited by 3,846
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  7. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  8. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  9. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  10. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  11. 2016Concrete Problems in AI Safetycited by 1,582
  12. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
38 more
  1. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  2. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  3. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
  4. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  5. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  6. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  7. 2018AI safety via debatecited by 384
  8. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  9. 2023AI Control: Improving Safety Despite Intentional Subversioncited by 207
  10. 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 174
  11. 2021Alignment of Language Agentscited by 167
  12. 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 162
  13. 2024Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviewscited by 160
  14. 2023Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigationscited by 157
  15. 2023Large Language Models can Strategically Deceive their Users when Put Under Pressurecited by 132
  16. 2024Evaluating Frontier Models for Dangerous Capabilitiescited by 121
  17. 2024Language Models Learn to Mislead Humans via RLHFcited by 118
  18. 2021Truthful AI: Developing and governing AI that does not liecited by 109
  19. 2016Winning Arguments: Interaction Dynamics and Persuasion Strategies in Good-faith Online Discussionscited by 106
  20. 2024On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trialcited by 105
  21. 2022Human heuristics for AI-generated language are flawedcited by 104
  22. 2023Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplationcited by 102
  23. 2022What Artificial Neural Networks Can Tell Us About Human Language Acquisitioncited by 97
  24. 2014The Rise of Social Botscited by 96
  25. 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Modelscited by 95
  26. 2023Evaluating Language-Model Agents on Realistic Autonomous Taskscited by 94
  27. 2024How Well Can LLMs Negotiate? NegotiationArena Platform and Analysiscited by 93
  28. 2023How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questionscited by 82
  29. 2023AI model GPT-3 (dis)informs us better than humanscited by 75
  30. 2023How FaR Are Large Language Models From Agents with Theory-of-Mind?cited by 75
  31. 2024Hidden Persuaders: LLMs' Political Leaning and Their Influence on Voterscited by 71
  32. 2023Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4cited by 67
  33. 2019A Large-scale Dataset for Argument Quality Ranking: Construction and Analysiscited by 66
  34. 2023Boosting Theory-of-Mind Performance in Large Language Models via Promptingcited by 64
  35. 2022Do Large Language Models know what humans know?cited by 60
  36. 2023Hoodwinked: Deception and Cooperation in a Text-Based Game for Language Modelscited by 46
  37. 2024A Safe Harbor for AI Evaluation and Red Teamingcited by 46
  38. 2024People cannot distinguish GPT-4 from a human in a Turing testcited by 46

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 4.

14 papers.

Sort:

By area: