# Investigating Memorization of Conspiracy Theories in Text Generation

arXiv: https://arxiv.org/abs/2101.00379 · HTML: https://mlsanity.com/paper/2101.00379/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 52 references that are in the map.
- First posted on arXiv 2 Jan 2021.

## Builds on (most cited first, 18 of 18)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://mlsanity.com/paper/1909.01326/) (2019)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation](https://mlsanity.com/paper/2008.03703/) (2020)
- [Mitigating Gender Bias in Natural Language Processing: Literature Review](https://mlsanity.com/paper/1906.08976/) (2019)
- [Incorporating Copying Mechanism in Sequence-to-Sequence Learning](https://mlsanity.com/paper/1603.06393/) (2016)
- [Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation](https://mlsanity.com/paper/1703.09902/) (2017)
- [The Radicalization Risks of GPT-3 and Advanced Neural Language Models](https://mlsanity.com/paper/2009.06807/) (2020)
- [Smart Reply: Automated Response Suggestion for Email](https://mlsanity.com/paper/1606.04870/) (2016)
- [Investigating African-American Vernacular English in Transformer-Based Text Generation](https://mlsanity.com/paper/2010.02510/) (2020)
- [A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk](https://mlsanity.com/paper/1712.05796/) (2017)
- [Science vs Conspiracy: collective narratives in the age of (mis)information](https://mlsanity.com/paper/1408.1667/) (2014)
- [Towards Understanding Sample Variance in Visually Grounded Language Generation: Evaluations and Observations](https://mlsanity.com/paper/2010.03644/) (2020)

## Built on by (most cited first, 15 of 15)

- [Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies](https://mlsanity.com/paper/2308.03188/) (2023)
- [Why Language Models Hallucinate](https://mlsanity.com/paper/2509.04664/) (2025)
- [Language Model Behavior: A Comprehensive Survey](https://mlsanity.com/paper/2303.11504/) (2023)
- [Do Language Models Plagiarize?](https://mlsanity.com/paper/2203.07618/) (2022)
- ["According to ...": Prompting Language Models Improves Quoting from Pre-Training Data](https://mlsanity.com/paper/2305.13252/) (2023)
- [Learning to Refine with Fine-Grained Natural Language Feedback](https://mlsanity.com/paper/2407.02397/) (2024)
- [Chain of Explanation: New Prompting Method to Generate Higher Quality Natural Language Explanation for Implicit Hate Speech](https://mlsanity.com/paper/2209.04889/) (2022)
- [Mitigating Covertly Unsafe Text within Natural Language Systems](https://mlsanity.com/paper/2210.09306/) (2022)
- [ROME: Memorization Insights from Text, Logits and Representation](https://mlsanity.com/paper/2403.00510/) (2024)
- [Generating Natural Language Queries for More Effective Systematic Review Screening Prioritisation](https://mlsanity.com/paper/2309.05238/) (2023)
- [Systematic Assessment of Factual Knowledge in Large Language Models](https://mlsanity.com/paper/2310.11638/) (2023)
- [What distinguishes conspiracy from critical narratives? A computational analysis of oppositional discourse](https://mlsanity.com/paper/2407.10745/) (2024)
- [ConspEmoLLM-v2: A robust and stable model to detect sentiment-transformed conspiracy theories](https://mlsanity.com/paper/2505.14917/) (2025)
- [ConspirED: A Dataset for Cognitive Traits of Conspiracy Theories and Large Language Model Safety](https://mlsanity.com/paper/2508.20468/) (2025)
- [Chatbots Output Meaningful (but Problematic) Language](https://mlsanity.com/paper/2606.02973/) (2026)

## Abstract (arXiv)

The adoption of natural language generation (NLG) models can leave individuals vulnerable to the generation of harmful information memorized by the models, such as conspiracy theories. While previous studies examine conspiracy theories in the context of social media, they have not evaluated their presence in the new space of generative language models. In this work, we investigate the capability of language models to generate conspiracy theory text. Specifically, we aim to answer: can we test pretrained generative language models for the memorization and elicitation of conspiracy theories without access to the model's training data? We highlight the difficulties of this task and discuss it in the context of memorization, generalization, and hallucination. Utilizing a new dataset consisting of conspiracy theory topics and machine-generated conspiracy theories helps us discover that many conspiracy theories are deeply rooted in the pretrained language models. Our experiments demonstrate a relationship between model parameters such as size and temperature and their propensity to generate conspiracy theory text. These results indicate the need for a more thorough review of NLG applications before release and an in-depth discussion of the drawbacks of memorization in generative language models.