# Who Decides What Is Harmful? Content Moderation Policy Through A Multi-Agent Personalised Inference Framework

arXiv: https://arxiv.org/abs/2605.01416 · HTML: https://mlsanity.com/paper/2605.01416/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 18 of its 41 references that are in the map.
- First posted on arXiv 2 May 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 18 of 18)

- [Mixture-of-Agents Enhances Large Language Model Capabilities](https://mlsanity.com/paper/2406.04692/) (2024)
- [Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models](https://mlsanity.com/paper/2510.04618/) (2025)
- [Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection](https://mlsanity.com/paper/2111.07997/) (2021)
- [SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)](https://mlsanity.com/paper/2006.07235/) (2020)
- [Toward a Perspectivist Turn in Ground Truthing for Predictive Computing](https://mlsanity.com/paper/2109.04270/) (2021)
- [Measuring a hate speech spectrum with faceted Rasch item response theory and perspective-aware, explainable-by-design deep learning](https://mlsanity.com/paper/2009.10277/) (2020)
- [The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labels](https://mlsanity.com/paper/2405.05860/) (2024)
- [Exploring Author Gender in Book Rating and Recommendation](https://mlsanity.com/paper/1808.07586/) (2018)
- [Robust Hate Speech Detection in Social Media: A Cross-Dataset Empirical Evaluation](https://mlsanity.com/paper/2307.01680/) (2023)
- [Hate Personified: Investigating the role of LLMs in content moderation](https://mlsanity.com/paper/2410.02657/) (2024)
- [Theory of Mind in Large Language Models: Assessment and Enhancement](https://mlsanity.com/paper/2505.00026/) (2025)
- [Personalizing Content Moderation on Social Media: User Perspectives on Moderation Choices, Interface Design, and Labor](https://mlsanity.com/paper/2305.10374/) (2023)
- [Are Large Language Models Sensitive to the Motives Behind Communication?](https://mlsanity.com/paper/2510.19687/) (2025)
- [Designing Word Filter Tools for Creator-led Comment Moderation](https://mlsanity.com/paper/2202.08818/) (2022)
- [Leveraging a Multi-Agent LLM-Based System to Educate Teachers in Hate Incidents Management](https://mlsanity.com/paper/2506.23774/) (2025)
- [Algorithmic Fairness in NLP: Persona-Infused LLMs for Human-Centric Hate Speech Detection](https://mlsanity.com/paper/2510.19331/) (2025)
- [A Natural Language Agentic Approach to Study Affective Polarization](https://mlsanity.com/paper/2603.02711/) (2026)
- [On Theoretically-Driven LLM Agents for Multi-Dimensional Discourse Analysis](https://mlsanity.com/paper/2602.13713/) (2026)

## Abstract (arXiv)

The increasing scale and complexity of online platforms raises critical policy questions around harmful content, digital well-being, and user autonomy. Traditional content moderation systems rely on centralised, top-down rules, often failing to accommodate the subjective nature of harm perception. This paper proposes an LLM-based multi-agent personalised inference framework that filters content based on unique sensitivity profiles of individual users. Our architecture combines domain-specific Expert Agents, a Manager Agent for orchestrating content analysis and agent selection, and a Ghost Profile Agent for simulating user perspectives, to inform moderation decisions. Evaluated against a range of non-personalised baselines, the system demonstrates up to a 32% improvement in accuracy, showing increased alignment with individual user sensitivities. Beyond technical performance, our framework provides policy-relevant insights for platform governance, providing a scalable way to reconcile moderation policies with societal and individual digital rights