# What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detection

arXiv: https://arxiv.org/abs/2402.00371 · HTML: https://mlsanity.com/paper/2402.00371/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 72 of its 108 references that are in the map.
- First posted on arXiv 1 Feb 2024.

## Builds on (most cited first, 30 of 72)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [Prometheus: Inducing Fine-grained Evaluation Capability in Language Models](https://mlsanity.com/paper/2310.08491/) (2023)
- [Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus](https://mlsanity.com/paper/2104.08758/) (2021)
- [MetaICL: Learning to Learn In Context](https://mlsanity.com/paper/2110.15943/) (2021)
- [From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models](https://mlsanity.com/paper/2305.08283/) (2023)
- [Can Language Models Solve Graph Problems in Natural Language?](https://mlsanity.com/paper/2305.10037/) (2023)
- [On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning](https://mlsanity.com/paper/2212.08061/) (2022)
- [Large Language Models on Graphs: A Comprehensive Survey](https://mlsanity.com/paper/2312.02783/) (2023)
- [Can LLM-Generated Misinformation Be Detected?](https://mlsanity.com/paper/2309.13788/) (2023)
- [Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speech](https://mlsanity.com/paper/2302.07736/) (2023)
- [Unlimiformer: Long-Range Transformers with Unlimited Length Input](https://mlsanity.com/paper/2305.01625/) (2023)
- [PRD: Peer Rank and Discussion Improve Large Language Model based Evaluations](https://mlsanity.com/paper/2307.02762/) (2023)
- [GCAN: Graph-aware Co-Attention Networks for Explainable Fake News Detection on Social Media](https://mlsanity.com/paper/2004.11648/) (2020)
- [The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation](https://mlsanity.com/paper/2312.09085/) (2023)
- [Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey](https://mlsanity.com/paper/2210.07700/) (2022)
- [Fake News in Sheep's Clothing: Robust Fake News Detection Against LLM-Empowered Style Attacks](https://mlsanity.com/paper/2310.10830/) (2023)
- [Demoting Racial Bias in Hate Speech Detection](https://mlsanity.com/paper/2005.12246/) (2020)
- [Scalable and Generalizable Social Bot Detection through Data Selection](https://mlsanity.com/paper/1911.09179/) (2019)
- [Probing LLMs for hate speech detection: strengths and vulnerabilities](https://mlsanity.com/paper/2310.12860/) (2023)

## Built on by (most cited first, 25 of 25)

- [Large Language Model Safety: A Holistic Survey](https://mlsanity.com/paper/2412.17686/) (2024)
- [Stumbling Blocks: Stress Testing the Robustness of Machine-Generated Text Detectors Under Attacks](https://mlsanity.com/paper/2402.11638/) (2024)
- [A Practical Examination of AI-Generated Text Detectors for Large Language Models](https://mlsanity.com/paper/2412.05139/) (2024)
- [A Comprehensive Analysis of Large Language Model Outputs: Similarity, Diversity, and Bias](https://mlsanity.com/paper/2505.09056/) (2025)
- [Consistency of Responses and Continuations Generated by Large Language Models on Social Media](https://mlsanity.com/paper/2501.08102/) (2025)
- [GuARD: Effective Anomaly Detection through a Text-Rich and Graph-Informed Language Model](https://mlsanity.com/paper/2412.03930/) (2024)
- [Large Language Models can Achieve Social Balance](https://mlsanity.com/paper/2410.04054/) (2024)
- [On the Risk of Evidence Pollution for Malicious Social Text Detection in the Era of LLMs](https://mlsanity.com/paper/2410.12600/) (2024)
- [How Sampling Affects the Detectability of Machine-written texts: A Comprehensive Study](https://mlsanity.com/paper/2510.13681/) (2025)
- [Sleeper Social Bots: a new generation of AI disinformation bots are already a political threat](https://mlsanity.com/paper/2408.12603/) (2024)
- [Static Sandboxes Are Inadequate: Modeling Societal Complexity Requires Open-Ended Co-Evolution in LLM-Based Multi-Agent Simulations](https://mlsanity.com/paper/2510.13982/) (2025)
- [TRACE-Bot: Detecting Emerging LLM-Driven Social Bots via Implicit Semantic Representations and AIGC-Enhanced Behavioral Patterns](https://mlsanity.com/paper/2604.02147/) (2026)
- [Detecting LLM-Generated Spam Reviews by Integrating Language Model Embeddings and Graph Neural Network](https://mlsanity.com/paper/2510.01801/) (2025)
- [RedOne 2.0: Rethinking Domain-specific LLM Post-Training in Social Networking Services](https://mlsanity.com/paper/2511.07070/) (2025)
- [What is a Social Media Bot? A Global Comparison of Bot and Human Characteristics](https://mlsanity.com/paper/2501.00855/) (2025)
- [How Large Language Models play humans in online conversations: a simulated study of the 2016 US politics on Reddit](https://mlsanity.com/paper/2506.21620/) (2025)
- [A Comparative Analysis of Transformer Models in Social Bot Detection](https://mlsanity.com/paper/2509.14936/) (2025)
- [Bot Meets Shortcut: How Can LLMs Aid in Handling Unknown Invariance OOD Scenarios?](https://mlsanity.com/paper/2511.08455/) (2025)
- [AgentRob: From Virtual Forum Agents to Hijacked Physical Robots](https://mlsanity.com/paper/2602.13591/) (2026)
- [Cyberbullying Governance on Social Media: A Unified Framework from Content Identification to Intervention](https://mlsanity.com/paper/2605.27584/) (2026)
- [SAHG: Sector-Anisotropic Hyperbolic Graph Model for Social Bot Detection](https://mlsanity.com/paper/2605.30166/) (2026)
- [Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerability](https://mlsanity.com/paper/2607.10402/) (2026)
- [On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Models](https://mlsanity.com/paper/2608.10530/) (2026)
- [Breaking and Defending LLM-Powered Social Media Bot Detection Systems](https://mlsanity.com/paper/2608.15893/) (2026)
- [CSC: Calibrated Simplicity for Conflict-Aware Social Bot Detection in the LLM Era](https://mlsanity.com/paper/2609.23320/) (2026)

## Abstract (arXiv)

Social media bot detection has always been an arms race between advancements in machine learning bot detectors and adversarial bot strategies to evade detection. In this work, we bring the arms race to the next level by investigating the opportunities and risks of state-of-the-art large language models (LLMs) in social bot detection. To investigate the opportunities, we design novel LLM-based bot detectors by proposing a mixture-of-heterogeneous-experts framework to divide and conquer diverse user information modalities. To illuminate the risks, we explore the possibility of LLM-guided manipulation of user textual and structured information to evade detection. Extensive experiments with three LLMs on two datasets demonstrate that instruction tuning on merely 1,000 annotated examples produces specialized LLMs that outperform state-of-the-art baselines by up to 9.1% on both datasets, while LLM-guided manipulation strategies could significantly bring down the performance of existing bot detectors by up to 29.6% and harm the calibration and reliability of bot detection systems.