# Emotional Manipulation Through Prompt Engineering Amplifies Disinformation Generation in AI Large Language Models

arXiv: https://arxiv.org/abs/2403.03550 · HTML: https://mlsanity.com/paper/2403.03550/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 17 references that are in the map.
- First posted on arXiv 6 Mar 2024.

## Built on by (most cited first, 5 of 5)

- [Reasoning Models Struggle to Control their Chains of Thought](https://mlsanity.com/paper/2603.05706/) (2026)
- [Emotions Where Art Thou: Understanding and Characterizing the Emotional Latent Space of Large Language Models](https://mlsanity.com/paper/2510.22042/) (2025)
- [ChatGPT Reads Your Tone and Responds Accordingly -- Until It Does Not -- Emotional Framing Induces Bias in LLM Outputs](https://mlsanity.com/paper/2507.21083/) (2025)
- [JailNewsBench: Multi-Lingual and Regional Benchmark for Fake News Generation under Jailbreak Attacks](https://mlsanity.com/paper/2603.01291/) (2026)
- [Measuring Pragmatic Influence in Large Language Model Instructions](https://mlsanity.com/paper/2602.21223/) (2026)

## Abstract (arXiv)

This study investigates the generation of synthetic disinformation by OpenAI's Large Language Models (LLMs) through prompt engineering and explores their responsiveness to emotional prompting. Leveraging various LLM iterations using davinci-002, davinci-003, gpt-3.5-turbo and gpt-4, we designed experiments to assess their success in producing disinformation. Our findings, based on a corpus of 19,800 synthetic disinformation social media posts, reveal that all LLMs by OpenAI can successfully produce disinformation, and that they effectively respond to emotional prompting, indicating their nuanced understanding of emotional cues in text generation. When prompted politely, all examined LLMs consistently generate disinformation at a high frequency. Conversely, when prompted impolitely, the frequency of disinformation production diminishes, as the models often refuse to generate disinformation and instead caution users that the tool is not intended for such purposes. This research contributes to the ongoing discourse surrounding responsible development and application of AI technologies, particularly in mitigating the spread of disinformation and promoting transparency in AI-generated content.