# Deception and Manipulation in Generative AI

arXiv: https://arxiv.org/abs/2401.11335 · HTML: https://mlsanity.com/paper/2401.11335/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- First posted on arXiv 20 Jan 2024.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Built on by (most cited first, 8 of 8)

- [AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agents](https://mlsanity.com/paper/2409.09013/) (2024)
- [A Causal Explainable Guardrails for Large Language Models](https://mlsanity.com/paper/2405.04160/) (2024)
- [Mechanistic Interpretability Needs Philosophy](https://mlsanity.com/paper/2506.18852/) (2025)
- [Watermarking Discrete Diffusion Language Models](https://mlsanity.com/paper/2511.02083/) (2025)
- [AI-Enhanced Deliberative Democracy and the Future of the Collective Will](https://mlsanity.com/paper/2503.05830/) (2025)
- [Detecting Malicious AI Agents Through Simulated Interactions](https://mlsanity.com/paper/2504.03726/) (2025)
- [Instrumental convergence and power-seeking](https://mlsanity.com/paper/2606.08832/) (2026)
- [Detecting AI-Generated Content on Social Media with Multi-modal Language Models](https://mlsanity.com/paper/2606.11200/) (2026)

## Abstract (arXiv)

Large language models now possess human-level linguistic abilities in many contexts. This raises the concern that they can be used to deceive and manipulate on unprecedented scales, for instance spreading political misinformation on social media. In future, agentic AI systems might also deceive and manipulate humans for their own ends. In this paper, first, I argue that AI-generated content should be subject to stricter standards against deception and manipulation than we ordinarily apply to humans. Second, I offer new characterizations of AI deception and manipulation meant to support such standards, according to which a statement is deceptive (manipulative) if it leads human addressees away from the beliefs (choices) they would endorse under ``semi-ideal'' conditions. Third, I propose two measures to guard against AI deception and manipulation, inspired by this characterization: "extreme transparency" requirements for AI-generated content and defensive systems that, among other things, annotate AI-generated statements with contextualizing information. Finally, I consider to what extent these measures can protect against deceptive behavior in future, agentic AIs, and argue that non-agentic defensive systems can provide an important layer of defense even against more powerful agentic systems.