# Characterizing Bluesky Content Moderation Service: From Automation of Service to Landscape of Harms

arXiv: https://arxiv.org/abs/2609.11373 · HTML: https://mlsanity.com/paper/2609.11373/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 28 references that are in the map.
- First posted on arXiv 10 Sep 2026.

## Builds on (most cited first, 10 of 10)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Ranking](https://mlsanity.com/paper/2601.04720/) (2026)
- [Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic Variations](https://mlsanity.com/paper/2503.01623/) (2025)
- ["I'm in the Bluesky Tonight": Insights from a Year Worth of Social Data](https://mlsanity.com/paper/2404.18984/) (2024)
- [Bluesky and the AT Protocol: Usable Decentralized Social Media](https://mlsanity.com/paper/2402.03239/) (2024)
- [A Year of the DSA Transparency Database: What it (Does Not) Reveal About Platform Moderation During the 2024 European Parliament Election](https://mlsanity.com/paper/2504.06976/) (2025)
- [Challenges in the Decentralised Web: The Mastodon Case](https://mlsanity.com/paper/1909.05801/) (2019)
- [Watching the Watchers: A Comparative Fairness Audit of Cloud-based Content Moderation Services](https://mlsanity.com/paper/2406.14154/) (2024)
- [BEYONDWORDS is All You Need: Agentic Generative AI based Social Media Themes Extractor](https://mlsanity.com/paper/2503.01880/) (2025)
- [Can Foundation Models Moderate Online Content? Evaluating Instruction- vs. Example-Driven Policy Operationalization](https://mlsanity.com/paper/2609.10410/) (2026)

## Abstract (arXiv)

Empirical research on content moderation is fundamentally constrained by the opaque deployment of moderation systems on major social media platforms. To this end, the recent emergence of decentralized platforms with transparent, public moderation logs presents an unprecedented opportunity for independent audits. In this work, we leverage this architectural transparency to conduct the first large-scale audit of the default moderation system on Bluesky, the Bluesky Moderation Service (BMS). Analyzing its 10.6M moderation labels from 2025, we investigate three foundational aspects: (i) its mechanism (the degree of automation versus human oversight), (ii) its efficacy (accuracy in detecting harms), and (iii) its purpose (the landscape of harms it identifies). Our findings reveal a human-AI collaborative system where labels for sexual and graphic content are applied automatically in seconds, while nuanced and high stakes labels require more human oversight, taking hours or days. Through a manual annotation study, we find the BMS operates with high precision (0.837), but struggles with low recall (0.222), with our annotators identifying 4.5× more harmful content than the moderation system in a random sample. Finally, unsupervised clustering of the most frequently applied labeled posts uncovers detected harms ranging from hostility in discourse toward protected groups to the spread of sexually explicit and other graphic content. Our work offers a look into the operational realities of a deployed moderation system, providing a concrete data-driven foundation for designing more effective and transparent moderation systems.