mlsanityData to 7 Oct 2026

Assessing the Impact of Conspiracy Theories Using Large Language Models

arXiv 2412.07019 ↗First posted 9 Dec 2024cs.CL cs.CYcited by 2

Not on the map. Why?

Abstract (arXiv)

Measuring the relative impact of CTs is important for prioritizing responses and allocating resources effectively, especially during crises. However, assessing the actual impact of CTs on the public poses unique challenges. It requires not only the collection of CT-specific knowledge but also diverse information from social, psychological, and cultural dimensions. Recent advancements in large language models (LLMs) suggest their potential utility in this context, not only due to their extensive knowledge from large training corpora but also because they can be harnessed for complex reasoning. In this work, we develop datasets of popular CTs with human-annotated impacts. Borrowing insights from human impact assessment processes, we then design tailored strategies to leverage LLMs for performing human-like CT impact assessments. Through rigorous experiments, we textit{discover that an impact assessment mode using multi-step reasoning to analyze more CT-related evidence critically produces accurate results; and most LLMs demonstrate strong bias, such as assigning higher impacts to CTs presented earlier in the prompt, while generating less accurate impact assessments for emotionally charged and verbose CTs.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 31 of its 65 references that are in the map. First posted on arXiv 9 Dec 2024.

Builds on

31 of 65 references.

References · 31

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  7. 2024Qwen2 Technical Reportcited by 2,076
  8. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  9. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 416
  10. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
  11. 2023S3: Social-network Simulation System with Large Language Model-Empowered Agentscited by 228
  12. 2023Can large language models provide useful feedback on research papers? A large-scale empirical analysiscited by 195
19 more
  1. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  2. 2023Benchmarking Foundation Models with Language-Model-as-an-Examinercited by 172
  3. 2024LLM Critics Help Catch LLM Bugscited by 138
  4. 2023Human-like Summarization Evaluation with ChatGPTcited by 126
  5. 2023ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewingcited by 118
  6. 2023The science of fake newscited by 94
  7. 2024DALK: Dynamic Co-Augmentation of LLMs and KG to answer Alzheimer's Disease Questions with Scientific Literaturecited by 50
  8. 2023Disinformation Detection: An Evolving Challenge in the Age of LLMscited by 41
  9. 2024A Taxonomy for Human-LLM Interaction Modes: An Initial Explorationcited by 38
  10. 2024CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding?cited by 29
  11. 2024Social Science Meets LLMs: How Reliable Are Large Language Models in Social Simulations?cited by 26
  12. 2024BPO: Towards Balanced Preference Optimization between Knowledge Breadth and Depth in Alignmentcited by 20
  13. 2024Can LLMs Improve Multimodal Fact-Checking by Asking Relevant Questions?cited by 17
  14. 2024LLM-RadJudge: Achieving Radiologist-Level Evaluation for X-Ray Report Generationcited by 14
  15. 2023YouNICon: YouTube's CommuNIty of Conspiracy Videoscited by 13
  16. 2024PopALM: Popularity-Aligned Language Models for Social Media Trendy Response Predictioncited by 8
  17. 2021Mechanisms and Attributes of Echo Chambers in Social Mediacited by 5
  18. 2024Classifying Conspiratorial Narratives At Scale: False Alarms and Erroneous Connectionscited by 5
  19. 2023Online conspiracy communities are more resilient to deplatformingcited by 1

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Preference Leakage: A Contamination Problem in LLM-as-a-judgecited by 119
  2. 2025Who's Your Judge? On the Detectability of LLM-Generated Judgmentscited by 6