mlsanityData to 14 Oct 2026

MoMoE: Mixture of Moderation Experts Framework for AI-Assisted Online Governance

arXiv 2505.14483 ↗First posted 20 May 2025cs.CLcited by 13

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

Large language models (LLMs) have shown great potential in flagging harmful content in online communities. Yet, existing approaches for moderation require a separate model for every community and are opaque in their decision-making, limiting real-world adoption. We introduce Mixture of Moderation Experts (MoMoE), a modular, cross-community framework that adds post-hoc explanations to scalable content moderation. MoMoE orchestrates four operators -- Allocate, Predict, Aggregate, Explain -- and is instantiated as seven community-specialized experts (MoMoE-Community) and five norm-violation experts (MoMoE-NormVio). On 30 unseen subreddits, the best variants obtain Micro-F1 scores of 0.72 and 0.67, respectively, matching or surpassing strong fine-tuned baselines while consistently producing concise and reliable explanations. Although community-specialized experts deliver the highest peak accuracy, norm-violation experts provide steadier performance across domains. These findings show that MoMoE yields scalable, transparent moderation without needing per-community fine-tuning. More broadly, they suggest that lightweight, explainable expert ensembles can guide future NLP and HCI research on trustworthy human-AI governance of online communities.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 21 of its 54 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

21 of 54 references.

References · 21

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2019A Just and Comprehensive Strategy for Using NLP to Address Online Abusecited by 49
  6. 2025Policy-as-Prompt: Rethinking Content Moderation in the Age of Large Language Modelscited by 31
  7. 2024Content Moderation by LLM: From Accuracy to Legitimacycited by 30
  8. 2024SLM-Mod: Small Language Models Surpass LLMs at Content Moderationcited by 27
  9. 2021Conversations Gone Alright: Quantifying and Predicting Prosocial Outcomes in Online Conversationscited by 20
  10. 2021Detecting Community Sensitive Norm Violations in Online Conversationscited by 18
  11. 2023Moderating New Waves of Online Hate with Chain-of-Thought Reasoning in Large Language Modelscited by 18
  12. 2025From Text to Trust: Empowering AI-assisted Decision Making with Adaptive LLM-powered Analysiscited by 9
9 more
  1. 2024Seeing Like an AI: How LLMs Apply (and Misapply) Wikipedia Neutrality Normscited by 8
  2. 2022Designing Word Filter Tools for Creator-led Comment Moderationcited by 7
  3. 2024Venire: A Machine Learning-Guided Panel Review System for Community Content Moderationcited by 7
  4. 2022Measuring the Prevalence of Anti-Social Behavior in Online Communitiescited by 6
  5. 2022Measuring the Monetary Value of Online Volunteer Workcited by 5
  6. 2025LLM-C3MOD: A Human-LLM Collaborative System for Cross-Cultural Hate Speech Moderationcited by 5
  7. 2024Uncovering the Internet's Hidden Values: An Empirical Study of Desirable Behavior Using Highly-Upvoted Content on Redditcited by 4
  8. 2019Hybrid Approaches to Detect Comments Violating Macro Norms on Redditcited by 2
  9. 2025Supporting AI-Augmented Meta-Decision Making with InDecisioncited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 12.

13 papers.

Sort:

By area:

  1. 2026RubRIX: Rubric-Driven Risk Mitigation in Caregiver-AI Interactionscited by 10
  2. 2026PAIR-SAFE: A Paired-Agent Approach for Runtime Auditing and Refining AI-Mediated Mental Health Supportcited by 7
  3. 2026Social Simulacra in the Wild: AI Agent Communities on Moltbookcited by 4
  4. 2025Evaluating Large Language Models for Detecting Antisemitismcited by 3
  5. 2026The Gray Area: Characterizing Moderator Disagreement on Redditcited by 3
  6. 2026From Plausible to Causal: Counterfactual Semantics for Policy Evaluation in Simulated Online Communitiescited by 2
  7. 2026LLUMI: Improving LLM Writing Assistance for Mental Health Support with Online Community Feedbackcited by 2
  8. 2026GMP: A Benchmark for Content Moderation under Co-occurring Violations and Dynamic Rulescited by 0
  9. 2026Masking or Mitigating? Deconstructing the Impact of Query Rewriting on Retriever Biases in RAGcited by 0
  10. 2026AI Content Moderation in Therapy Conversationscited by 0
  11. 2026When AI Says "I have been in similar situations": Synthetic Lived Experience in Peer-Like Caregiver Supportcited by 0
  12. 2026You Frame It: How Conceptual Representations Shape LLM Detection and Reasoning about Antisemitismcited by 0
1 more
  1. 2026Facts Without Rules: Boundary Metadata Collapse in Multi-Agent LLM Handoffscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026CHAIRO: Contextual Hierarchical Analogical Induction and Reasoning Optimization for LLMs
  2. 2024Safeguarding Decentralized Social Media: LLM Agents for Automating Community Rule Compliance
  3. 2026PluRule: A Benchmark for Moderating Pluralistic Communities on Social Media
  4. 2026Tool-MCoT: Tool Augmented Multimodal Chain-of-Thought for Content Safety Moderation
  5. 2025Watching the AI Watchdogs: A Fairness and Robustness Analysis of AI Safety Moderation Classifiers
  6. 2023Adapting Large Language Models for Content Moderation: Pitfalls in Data Engineering and Supervised Fine-tuning
  7. 2026VASTU: Language Models Struggle to Recognize Online Community Values
  8. 2026Can Foundation Models Moderate Online Content? Evaluating Instruction- vs. Example-Driven Policy Operationalization