# MoMoE: Mixture of Moderation Experts Framework for AI-Assisted Online Governance

arXiv: https://arxiv.org/abs/2505.14483 · HTML: https://mlsanity.com/paper/2505.14483/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security.
- Builds on 21 of its 54 references that are in the map.
- First posted on arXiv 20 May 2025.

## Placed in

- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)

## Builds on (most cited first, 21 of 21)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [A Just and Comprehensive Strategy for Using NLP to Address Online Abuse](https://mlsanity.com/paper/1906.01738/) (2019)
- [Policy-as-Prompt: Rethinking Content Moderation in the Age of Large Language Models](https://mlsanity.com/paper/2502.18695/) (2025)
- [Content Moderation by LLM: From Accuracy to Legitimacy](https://mlsanity.com/paper/2409.03219/) (2024)
- [SLM-Mod: Small Language Models Surpass LLMs at Content Moderation](https://mlsanity.com/paper/2410.13155/) (2024)
- [Conversations Gone Alright: Quantifying and Predicting Prosocial Outcomes in Online Conversations](https://mlsanity.com/paper/2102.08368/) (2021)
- [Detecting Community Sensitive Norm Violations in Online Conversations](https://mlsanity.com/paper/2110.04419/) (2021)
- [Moderating New Waves of Online Hate with Chain-of-Thought Reasoning in Large Language Models](https://mlsanity.com/paper/2312.15099/) (2023)
- [From Text to Trust: Empowering AI-assisted Decision Making with Adaptive LLM-powered Analysis](https://mlsanity.com/paper/2502.11919/) (2025)
- [Seeing Like an AI: How LLMs Apply (and Misapply) Wikipedia Neutrality Norms](https://mlsanity.com/paper/2407.04183/) (2024)
- [Designing Word Filter Tools for Creator-led Comment Moderation](https://mlsanity.com/paper/2202.08818/) (2022)
- [Venire: A Machine Learning-Guided Panel Review System for Community Content Moderation](https://mlsanity.com/paper/2410.23448/) (2024)
- [Measuring the Prevalence of Anti-Social Behavior in Online Communities](https://mlsanity.com/paper/2208.13094/) (2022)
- [Measuring the Monetary Value of Online Volunteer Work](https://mlsanity.com/paper/2205.14528/) (2022)
- [LLM-C3MOD: A Human-LLM Collaborative System for Cross-Cultural Hate Speech Moderation](https://mlsanity.com/paper/2503.07237/) (2025)
- [Uncovering the Internet's Hidden Values: An Empirical Study of Desirable Behavior Using Highly-Upvoted Content on Reddit](https://mlsanity.com/paper/2410.13036/) (2024)
- [Hybrid Approaches to Detect Comments Violating Macro Norms on Reddit](https://mlsanity.com/paper/1904.03596/) (2019)
- [Supporting AI-Augmented Meta-Decision Making with InDecision](https://mlsanity.com/paper/2504.12433/) (2025)

## Built on by (most cited first, 13 of 13)

- [RubRIX: Rubric-Driven Risk Mitigation in Caregiver-AI Interactions](https://mlsanity.com/paper/2601.13235/) (2026)
- [PAIR-SAFE: A Paired-Agent Approach for Runtime Auditing and Refining AI-Mediated Mental Health Support](https://mlsanity.com/paper/2601.12754/) (2026)
- [Social Simulacra in the Wild: AI Agent Communities on Moltbook](https://mlsanity.com/paper/2603.16128/) (2026)
- [Evaluating Large Language Models for Detecting Antisemitism](https://mlsanity.com/paper/2509.18293/) (2025)
- [The Gray Area: Characterizing Moderator Disagreement on Reddit](https://mlsanity.com/paper/2601.01620/) (2026)
- [From Plausible to Causal: Counterfactual Semantics for Policy Evaluation in Simulated Online Communities](https://mlsanity.com/paper/2604.03920/) (2026)
- [LLUMI: Improving LLM Writing Assistance for Mental Health Support with Online Community Feedback](https://mlsanity.com/paper/2605.30273/) (2026)
- [GMP: A Benchmark for Content Moderation under Co-occurring Violations and Dynamic Rules](https://mlsanity.com/paper/2603.01724/) (2026)
- [Masking or Mitigating? Deconstructing the Impact of Query Rewriting on Retriever Biases in RAG](https://mlsanity.com/paper/2604.06097/) (2026)
- [AI Content Moderation in Therapy Conversations](https://mlsanity.com/paper/2605.25454/) (2026)
- [When AI Says "I have been in similar situations": Synthetic Lived Experience in Peer-Like Caregiver Support](https://mlsanity.com/paper/2606.18057/) (2026)
- [You Frame It: How Conceptual Representations Shape LLM Detection and Reasoning about Antisemitism](https://mlsanity.com/paper/2607.04945/) (2026)
- [Facts Without Rules: Boundary Metadata Collapse in Multi-Agent LLM Handoffs](https://mlsanity.com/paper/2608.29028/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have shown great potential in flagging harmful content in online communities. Yet, existing approaches for moderation require a separate model for every community and are opaque in their decision-making, limiting real-world adoption. We introduce Mixture of Moderation Experts (MoMoE), a modular, cross-community framework that adds post-hoc explanations to scalable content moderation. MoMoE orchestrates four operators -- Allocate, Predict, Aggregate, Explain -- and is instantiated as seven community-specialized experts (MoMoE-Community) and five norm-violation experts (MoMoE-NormVio). On 30 unseen subreddits, the best variants obtain Micro-F1 scores of 0.72 and 0.67, respectively, matching or surpassing strong fine-tuned baselines while consistently producing concise and reliable explanations. Although community-specialized experts deliver the highest peak accuracy, norm-violation experts provide steadier performance across domains. These findings show that MoMoE yields scalable, transparent moderation without needing per-community fine-tuning. More broadly, they suggest that lightweight, explainable expert ensembles can guide future NLP and HCI research on trustworthy human-AI governance of online communities.