# Can Language Model Moderators Improve the Health of Online Discourse?

arXiv: https://arxiv.org/abs/2311.10781 · HTML: https://mlsanity.com/paper/2311.10781/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 52 references that are in the map.
- First posted on arXiv 16 Nov 2023.

## Builds on (most cited first, 22 of 22)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://mlsanity.com/paper/2304.07327/) (2023)
- [Identifying and Reducing Gender Bias in Word-Level Language Models](https://mlsanity.com/paper/1904.03035/) (2019)
- [SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization](https://mlsanity.com/paper/2212.10465/) (2022)
- [ProsocialDialog: A Prosocial Backbone for Conversational Agents](https://mlsanity.com/paper/2205.12688/) (2022)
- [ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons](https://mlsanity.com/paper/1909.03087/) (2019)
- [Mitigating Political Bias in Language Models Through Reinforced Calibration](https://mlsanity.com/paper/2104.14795/) (2021)
- [The Wisdom of Hindsight Makes Language Models Better Instruction Followers](https://mlsanity.com/paper/2302.05206/) (2023)
- [Civil Rephrases Of Toxic Texts With Self-Supervised Transformers](https://mlsanity.com/paper/2102.05456/) (2021)
- [Detecting Community Sensitive Norm Violations in Online Conversations](https://mlsanity.com/paper/2110.04419/) (2021)
- [Towards Automatic Evaluation of Dialog Systems: A Model-Free Off-Policy Evaluation Approach](https://mlsanity.com/paper/2102.10242/) (2021)
- [Reconsidering Tweets: Intervening During Tweet Creation Decreases Offensive Content](https://mlsanity.com/paper/2112.00773/) (2021)
- [Content Removal as a Moderation Strategy: Compliance and Other Outcomes in the ChangeMyView Community](https://mlsanity.com/paper/1910.09563/) (2019)
- [Echo Chambers on Social Media: A comparative analysis](https://mlsanity.com/paper/2004.09603/) (2020)

## Built on by (most cited first, 12 of 12)

- [Real or Robotic? Assessing Whether LLMs Accurately Simulate Qualities of Human Responses in Dialogue](https://mlsanity.com/paper/2409.08330/) (2024)
- [Contextualized Counterspeech: Strategies for Adaptation, Personalization, and Evaluation](https://mlsanity.com/paper/2412.07338/) (2024)
- [Evaluation and Facilitation of Online Discussions in the LLM Era: A Survey](https://mlsanity.com/paper/2503.01513/) (2025)
- [KODIS: A Multicultural Dispute Resolution Dialogue Corpus](https://mlsanity.com/paper/2504.12723/) (2025)
- [OATH-Frames: Characterizing Online Attitudes Towards Homelessness with LLM Assistants](https://mlsanity.com/paper/2406.14883/) (2024)
- [Promoting Constructive Deliberation: Reframing for Receptiveness](https://mlsanity.com/paper/2405.15067/) (2024)
- [Towards Effective Counter-Responses: Aligning Human Preferences with Strategies to Combat Online Trolling](https://mlsanity.com/paper/2410.04164/) (2024)
- [Designing Synthetic Discussion Generation Systems: A Case Study for Online Facilitation](https://mlsanity.com/paper/2503.16505/) (2025)
- [BotEval: Facilitating Interactive Human Evaluation](https://mlsanity.com/paper/2407.17770/) (2024)
- [Emotionally-Aware Agents for Dispute Resolution](https://mlsanity.com/paper/2509.04465/) (2025)
- [Illusions of the Gold Standard: A Large-scale Analysis of Human Evaluation Protocols for Long-form Text Generation](https://mlsanity.com/paper/2606.07936/) (2026)
- [Contextualized Counterspeech Can Be More Persuasive Than Generic Counterspeech](https://mlsanity.com/paper/2607.26236/) (2026)

## Abstract (arXiv)

Conversational moderation of online communities is crucial to maintaining civility for a constructive environment, but it is challenging to scale and harmful to moderators. The inclusion of sophisticated natural language generation modules as a force multiplier to aid human moderators is a tantalizing prospect, but adequate evaluation approaches have so far been elusive. In this paper, we establish a systematic definition of conversational moderation effectiveness grounded on moderation literature and establish design criteria for conducting realistic yet safe evaluation. We then propose a comprehensive evaluation framework to assess models' moderation capabilities independently of human intervention. With our framework, we conduct the first known study of language models as conversational moderators, finding that appropriately prompted models that incorporate insights from social science can provide specific and fair feedback on toxic behavior but struggle to influence users to increase their levels of respect and cooperation.