ConspirED: A Dataset for Cognitive Traits of Conspiracy Theories and Large Language Model Safety
Not on the map. Why?
Abstract (arXiv)
Conspiracy theories erode public trust in science and institutions while resisting debunking by evolving and absorbing counter-evidence. As AI-generated misinformation becomes increasingly sophisticated, understanding the rhetorical patterns in conspiratorial content is important for developing interventions such as targeted prebunking and assessing AI vulnerabilities. We introduce CONSPIRED (CONSPIR Evaluation Dataset), which captures the cognitive traits of conspiratorial ideation in multi-sentence excerpts (80-120 words) from online conspiracy articles, annotated using the CONSPIR cognitive framework. CONSPIRED is the first dataset of conspiratorial content annotated for general cognitive traits. Using CONSPIRED, we (i) develop computational models that identify conspiratorial traits and the dominant trait in text excerpts, and (ii) evaluate LLM robustness to conspiratorial inputs. We find that LLMs are readily misaligned by conspiratorial framing, reproducing its rhetorical patterns even when successfully deflecting comparable fact-checked misinformation.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 49 of its 93 references that are in the map. First posted on arXiv 28 Aug 2025.
Builds on
49 of 93 references.
References · 49
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
- 2016Concrete Problems in AI Safetycited by 1,642
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
- 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
- 2022Red Teaming Language Models with Language Modelscited by 1,026
37 more
- 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
- 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
- 2019Certified Data Removal from Machine Learning Modelscited by 535
- 2023Detecting Pretraining Data from Large Language Modelscited by 387
- 2021A Survey on Automated Fact-Checkingcited by 378
- 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 348
- 2021Time-Aware Language Models as Temporal Knowledge Basescited by 296
- 2022Data Contamination: From Memorization to Exploitationcited by 236
- 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 230
- 2022Efficient Few-Shot Learning Without Promptscited by 137
- 2019Fine-Grained Analysis of Propaganda in News Articlescited by 133
- 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 128
- 2024Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Modelscited by 122
- 2025QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learningcited by 50
- 2025Improving LLM Safety Alignment with Dual-Objective Optimizationcited by 43
- 2023Analyzing Dataset Annotation Quality Management in the Wildcited by 41
- 2021Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attentioncited by 40
- 2021MultiCite: Modeling realistic citations requires moving beyond the single-sentence single-label settingcited by 30
- 2024The Automated Verification of Textual Claims (AVeriTeC) Shared Taskcited by 29
- 2021Overview of the CLEF--2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake Newscited by 23
- 2024Evaluation of LLM Vulnerabilities to Being Misused for Personalized Disinformation Generationcited by 20
- 2023Large Language Models for Propaganda Span Annotationcited by 19
- 2025How to Protect Yourself from 5G Radiation? Investigating LLM Responses to Implicit Misinformationcited by 18
- 2020An automated pipeline for the discovery of conspiracy and conspiracy theory narrative frameworks: Bridgegate, Pizzagate and storytelling on the webcited by 16
- 2021Investigating Memorization of Conspiracy Theories in Text Generationcited by 15
- 2024Missci: Reconstructing Fallacies in Misrepresented Sciencecited by 14
- 2025Self-Destructive Language Modelcited by 10
- 2023Identifying Conspiracy Theories News based on Event Relation Graphcited by 8
- 2024Conspiracy theories and where to find them on TikTokcited by 7
- 2025Fine-grained Fallacy Detection with Human Label Variationcited by 6
- 2024Building Better Datasets: Seven Recommendations for Responsible Design from Dataset Creatorscited by 5
- 2022Which side are you on? Insider-Outsider classification in conspiracy-theoretic social mediacited by 4
- 2023MAFALDA: A Benchmark and Comprehensive Study of Fallacy Detection and Classificationcited by 4
- 2024Detection of Conspiracy Theories Beyond Keyword Bias in German-Language Telegram Using Large Language Modelscited by 4
- 2025Debunking with Dialogue? Exploring AI-Generated Counterspeech to Challenge Conspiracy Theoriescited by 2
- 2025ConspEmoLLM-v2: A robust and stable model to detect sentiment-transformed conspiracy theoriescited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.