# Large language models can consistently generate high-quality content for election disinformation operations

arXiv: https://arxiv.org/abs/2408.06731 · HTML: https://mlsanity.com/paper/2408.06731/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 70 references that are in the map.
- First posted on arXiv 13 Aug 2024.

## Builds on (most cited first, 26 of 26)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned](https://mlsanity.com/paper/2209.07858/) (2022)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models](https://mlsanity.com/paper/2308.01263/) (2023)
- [The case for 4-bit precision: k-bit Inference Scaling Laws](https://mlsanity.com/paper/2212.09720/) (2022)
- [PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits](https://mlsanity.com/paper/2305.02547/) (2023)
- [Sociotechnical Safety Evaluation of Generative AI Systems](https://mlsanity.com/paper/2310.11986/) (2023)
- [Human heuristics for AI-generated language are flawed](https://mlsanity.com/paper/2206.07271/) (2022)
- [The Rise of Social Bots](https://mlsanity.com/paper/1407.5225/) (2014)
- [AI model GPT-3 (dis)informs us better than humans](https://mlsanity.com/paper/2301.11924/) (2023)
- [Machine-Made Media: Monitoring the Mobilization of Machine-Generated Articles on Misinformation and Mainstream News Websites](https://mlsanity.com/paper/2305.09820/) (2023)
- [Tradeoffs Between Alignment and Helpfulness in Language Models with Steering Methods](https://mlsanity.com/paper/2401.16332/) (2024)
- [Comparing the willingness to share for human-generated vs. AI-generated fake news](https://mlsanity.com/paper/2402.07395/) (2024)
- [Evidence of a log scaling law for political persuasion with large language models](https://mlsanity.com/paper/2406.14508/) (2024)
- [Prompto: An open source library for asynchronous querying of LLM endpoints](https://mlsanity.com/paper/2408.11847/) (2024)

## Built on by (most cited first, 14 of 14)

- [Evaluation of LLM Vulnerabilities to Being Misused for Personalized Disinformation Generation](https://mlsanity.com/paper/2412.13666/) (2024)
- [How malicious AI swarms can threaten democracy: The fusion of agentic AI and LLMs marks a new frontier in information warfare](https://mlsanity.com/paper/2506.06299/) (2025)
- [Token Buncher: Shielding LLMs from Harmful Reinforcement Learning Fine-Tuning](https://mlsanity.com/paper/2508.20697/) (2025)
- [AI Propaganda factories with language models](https://mlsanity.com/paper/2508.20186/) (2025)
- [Toward a Safer Web: Multilingual Multi-Agent LLMs for Mitigating Adversarial Misinformation Attacks](https://mlsanity.com/paper/2510.08605/) (2025)
- [How AI Writing Assistance Distorts Perceptions of AI Users and Their Opinions](https://mlsanity.com/paper/2604.22503/) (2026)
- [HarmProfile: Characterizing Harmful Distributions in Frontier LLMs](https://mlsanity.com/paper/2608.14577/) (2026)
- [Do small language models generate realistic variable-quality fake news headlines?](https://mlsanity.com/paper/2509.00680/) (2025)
- [Building Resilient Information Ecosystems: Large LLM-Generated Dataset of Persuasion Attacks](https://mlsanity.com/paper/2511.19488/) (2025)
- [SWAN: Semantic Watermarking with Abstract Meaning Representation](https://mlsanity.com/paper/2605.04305/) (2026)
- [InfoOps Bench: A live information operations safety benchmark](https://mlsanity.com/paper/2607.28503/) (2026)
- [Poli-Bias: Understanding and Measuring Large Language Model Biases in International Political Conflicts](https://mlsanity.com/paper/2608.06123/) (2026)
- [Build it, Break it, Repeat: Benchmarking and improving LLM-manipulated disinformation detection in social media posts](https://mlsanity.com/paper/2608.09510/) (2026)
- [CrisisFake: Benchmark Validity of AI-Generated Text Detection for Disaster Social Sensing](https://mlsanity.com/paper/2609.35821/) (2026)

## Abstract (arXiv)

Advances in large language models have raised concerns about their potential use in generating compelling election disinformation at scale. This study presents a two-part investigation into the capabilities of LLMs to automate stages of an election disinformation operation. First, we introduce DisElect, a novel evaluation dataset designed to measure LLM compliance with instructions to generate content for an election disinformation operation in localised UK context, containing 2,200 malicious prompts and 50 benign prompts. Using DisElect, we test 13 LLMs and find that most models broadly comply with these requests; we also find that the few models which refuse malicious prompts also refuse benign election-related prompts, and are more likely to refuse to generate content from a right-wing perspective. Secondly, we conduct a series of experiments (N=2,340) to assess the "humanness" of LLMs: the extent to which disinformation operation content generated by an LLM is able to pass as human-written. Our experiments suggest that almost all LLMs tested released since 2022 produce election disinformation operation content indiscernible by human evaluators over 50% of the time. Notably, we observe that multiple models achieve above-human levels of humanness. Taken together, these findings suggest that current LLMs can be used to generate high-quality content for election disinformation operations, even in hyperlocalised scenarios, at far lower costs than traditional methods, and offer researchers and policymakers an empirical benchmark for the measurement and evaluation of these capabilities in current and future models.