Are Open-Weight LLMs Ready for Social Media Moderation? A Comparative Study on Bluesky
On the map: LLM safety and security → Safety alignment, refusal and guardrails
Abstract (arXiv)
As internet access expands, so does exposure to harmful content, increasing the need for effective moderation. Research has demonstrated that large language models (LLMs) can be effectively utilized for social media moderation tasks, including harmful content detection. While proprietary LLMs have been shown to zero-shot outperform traditional machine learning models, the out-of-the-box capability of open-weight LLMs remains an open question. Motivated by recent developments of reasoning LLMs, we evaluate seven state-of-the-art models: four proprietary and three open-weight. Testing with real-world posts on Bluesky, moderation decisions by Bluesky Moderation Service, and annotations by two authors, we find a considerable degree of overlap between the sensitivity (81%--97%) and specificity (91%--100%) of the open-weight LLMs and those (72%--98%, and 93%--99%) of the proprietary ones. Additionally, our analysis reveals that specificity exceeds sensitivity for rudeness detection, but the opposite holds for intolerance and threats. Lastly, we identify inter-rater agreement across human moderators and the LLMs, highlighting considerations for deploying LLMs in both platform-scale and personalized moderation contexts. These findings show open-weight LLMs can support privacy-preserving moderation on consumer-grade hardware and suggest new directions for designing moderation systems that balance community values with individual user preferences.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 12 of its 32 references that are in the map. First posted on arXiv 5 Feb 2026.
Builds on
12 of 32 references.
References · 12
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2022A New Generation of Perspective API: Efficient Multilingual Character-level Transformerscited by 181
- 2025NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Modelcited by 94
- 2024SLM-Mod: Small Language Models Surpass LLMs at Content Moderationcited by 27
- 2024What Should We Engineer in Prompts? Training Humans in Requirement-Driven LLM Usecited by 24
- 2024An Investigation of Large Language Models for Real-World Hate Speech Detectioncited by 20
- 2019Disturbed YouTube for Kids: Characterizing and Detecting Inappropriate Videos Targeting Young Childrencited by 13
- 2024Moderating Illicit Online Image Promotion for Unsafe User-Generated Content Games Using Large Vision-Language Modelscited by 13
- 2024Open LLMs are Necessary for Current Private Adaptations and Outperform their Closed Alternativescited by 12
- 2024Bluesky and the AT Protocol: Usable Decentralized Social Mediacited by 9
- 2024Safeguarding Decentralized Social Media: LLM Agents for Automating Community Rule Compliancecited by 5
- 2024Looking AT the Blue Skies of Blueskycited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Evaluating Large Language Models for Detecting Antisemitism
- 2023Adapting Large Language Models for Content Moderation: Pitfalls in Data Engineering and Supervised Fine-tuning
- 2024Rethinking Scale: The Efficacy of Fine-Tuned Open-Source LLMs in Large-Scale Reproducible Social Science Research
- 2024Recent Advances in Hate Speech Moderation: Multimodality and the Role of Large Models
- 2024Eliciting Uncertainty in Chain-of-Thought to Mitigate Bias against Forecasting Harmful User Behaviors
- 2025Are LLMs Enough for Hyperpartisan, Fake, Polarized and Harmful Content Detection? Evaluating In-Context Learning vs. Fine-Tuning
- 2024Improving and Assessing the Fidelity of Large Language Models Alignment to Online Communities
- 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment