Who Decides What Is Harmful? Content Moderation Policy Through A Multi-Agent Personalised Inference Framework
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
The increasing scale and complexity of online platforms raises critical policy questions around harmful content, digital well-being, and user autonomy. Traditional content moderation systems rely on centralised, top-down rules, often failing to accommodate the subjective nature of harm perception. This paper proposes an LLM-based multi-agent personalised inference framework that filters content based on unique sensitivity profiles of individual users. Our architecture combines domain-specific Expert Agents, a Manager Agent for orchestrating content analysis and agent selection, and a Ghost Profile Agent for simulating user perspectives, to inform moderation decisions. Evaluated against a range of non-personalised baselines, the system demonstrates up to a 32% improvement in accuracy, showing increased alignment with individual user sensitivities. Beyond technical performance, our framework provides policy-relevant insights for platform governance, providing a scalable way to reconcile moderation policies with societal and individual digital rights
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 18 of its 41 references that are in the map. First posted on arXiv 2 May 2026.
Builds on
18 of 41 references.
References · 18
- 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 420
- 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 349
- 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 258
- 2020SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)cited by 138
- 2021Toward a Perspectivist Turn in Ground Truthing for Predictive Computingcited by 129
- 2020Measuring a hate speech spectrum with faceted Rasch item response theory and perspective-aware, explainable-by-design deep learningcited by 67
- 2024The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labelscited by 37
- 2018Exploring Author Gender in Book Rating and Recommendationcited by 29
- 2023Robust Hate Speech Detection in Social Media: A Cross-Dataset Empirical Evaluationcited by 28
- 2025Theory of Mind in Large Language Models: Assessment and Enhancementcited by 24
- 2024Hate Personified: Investigating the role of LLMs in content moderationcited by 23
- 2023Personalizing Content Moderation on Social Media: User Perspectives on Moderation Choices, Interface Design, and Laborcited by 16
6 more
- 2022Designing Word Filter Tools for Creator-led Comment Moderationcited by 7
- 2025Are Large Language Models Sensitive to the Motives Behind Communication?cited by 7
- 2025Leveraging a Multi-Agent LLM-Based System to Educate Teachers in Hate Incidents Managementcited by 3
- 2025Algorithmic Fairness in NLP: Persona-Infused LLMs for Human-Centric Hate Speech Detectioncited by 2
- 2026A Natural Language Agentic Approach to Study Affective Polarizationcited by 2
- 2026On Theoretically-Driven LLM Agents for Multi-Dimensional Discourse Analysiscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Policy-as-Prompt: Rethinking Content Moderation in the Age of Large Language Models
- 2026SovereignPA-Bench: Evaluating User-Owned Personal Agents under Evolving Intent, Platform Mediation, and Consent Constraints
- 2025Aetheria: A multimodal interpretable content safety framework based on multi-agent debate and collaboration
- 2026Escaping the Agreement Trap: Defensibility Signals for Evaluating Rule-Governed AI
- 2025MV-Debate: Multi-view Agent Debate with Dynamic Reflection Gating for Multimodal Harmful Content Detection in Social Media
- 2026Open-Domain Safety Policy Construction
- 2025Towards Trustworthy Multimodal Moderation via Policy-Aligned Reasoning and Hierarchical Labeling
- 2026Differential Harm Propensity in Personalized LLM Agents: The Curious Case of Mental Health Disclosure