# Evaluating the Propensity of Generative AI for Producing Harmful Disinformation During the 2024 US Election Cycle

arXiv: https://arxiv.org/abs/2411.06120 · HTML: https://mlsanity.com/paper/2411.06120/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 41 references that are in the map.
- First posted on arXiv 9 Nov 2024.

## Builds on (most cited first, 10 of 10)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned](https://mlsanity.com/paper/2209.07858/) (2022)
- [Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMs](https://mlsanity.com/paper/2308.13387/) (2023)
- [The spread of low-credibility content by social bots](https://mlsanity.com/paper/1707.07592/) (2017)
- [Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models](https://mlsanity.com/paper/2206.08325/) (2022)
- [Bots increase exposure to negative and inflammatory content in online social systems](https://mlsanity.com/paper/1802.07292/) (2018)
- [Trends in the Diffusion of Misinformation on Social Media](https://mlsanity.com/paper/1809.05901/) (2018)
- [Comparing the willingness to share for human-generated vs. AI-generated fake news](https://mlsanity.com/paper/2402.07395/) (2024)
- [The role of bot squads in the political propaganda on Twitter](https://mlsanity.com/paper/1905.12687/) (2019)
- [Studying Fake News Spreading, Polarisation Dynamics, and Manipulation by Bots: a Tale of Networks and Language](https://mlsanity.com/paper/2109.07909/) (2021)

## Abstract (arXiv)

Generative Artificial Intelligence offers a powerful tool for adversaries who wish to engage in influence operations, such as the Chinese Spamouflage operation and the Russian Internet Research Agency effort that both sought to interfere with recent US election cycles. Therefore, this study seeks to investigate the propensity of current generative AI models for producing harmful disinformation during an election cycle. The probability that different generative AI models produced disinformation when given adversarial prompts was evaluated, in addition to the associated harm. This allows for the expected harm for each model to be computed and it was discovered that Copilot and Gemini tied for the overall safest performance by realizing the lowest expected harm, while GPT-4o produced the greatest rates of harmful disinformation, resulting in much higher expected harm scores. The impact of disinformation category was also investigated and Gemini was safest within the political category of disinformation due to mitigation attempts made by developers during the election, while Copilot was safest for topics related to health. Moreover, characteristics of adversarial roles were discovered that led to greater expected harm across all models. Finally, classification models were developed that predicted disinformation production based on the conditions considered in this study, which offers insight into factors important for predicting disinformation production. Based on all of these insights, recommendations are provided that seek to mitigate factors that lead to harmful disinformation being produced by generative AI models. It is hoped that developers will use these insights to improve future models.