# AI vs. Human Judgment of Content Moderation: LLM-as-a-Judge and Ethics-Based Response Refusals

arXiv: https://arxiv.org/abs/2505.15365 · HTML: https://mlsanity.com/paper/2505.15365/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 2 references that are in the map.
- First posted on arXiv 21 May 2025.

## Builds on (most cited first, 1 of 1)

- [The Ethics of AI Ethics -- An Evaluation of Guidelines](https://mlsanity.com/paper/1903.03425/) (2019)

## Built on by (most cited first, 5 of 5)

- [Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges](https://mlsanity.com/paper/2507.19672/) (2025)
- [Automated Concept Discovery for LLM-as-a-Judge Preference Analysis](https://mlsanity.com/paper/2603.03319/) (2026)
- [From Refusal Tokens to Refusal Control: Discovering and Steering Category-Specific Refusal Directions](https://mlsanity.com/paper/2603.13359/) (2026)
- [LLMs Judge Themselves: A Game-Theoretic Framework for Human-Aligned Evaluation](https://mlsanity.com/paper/2510.15746/) (2025)
- [Manac'a-1B-Instruct-2: Reproducible Safety Alignment for a Native Brazilian-Portuguese LLM. Negative DPO Results, a Controlled Composition Scan, and a Silent Failure Mode of the LLM Judge](https://mlsanity.com/paper/2610.20570/) (2026)

## Abstract (arXiv)

As large language models (LLMs) are increasingly deployed in high-stakes settings, their ability to refuse ethically sensitive prompts-such as those involving hate speech or illegal activities-has become central to content moderation and responsible AI practices. While refusal responses can be viewed as evidence of ethical alignment and safety-conscious behavior, recent research suggests that users may perceive them negatively. At the same time, automated assessments of model outputs are playing a growing role in both evaluation and training. In particular, LLM-as-a-Judge frameworks-in which one model is used to evaluate the output of another-are now widely adopted to guide benchmarking and fine-tuning. This paper examines whether such model-based evaluators assess refusal responses differently than human users. Drawing on data from Chatbot Arena and judgments from two AI judges (GPT-4o and Llama 3 70B), we compare how different types of refusals are rated. We distinguish ethical refusals, which explicitly cite safety or normative concerns (e.g., "I can't help with that because it may be harmful"), and technical refusals, which reflect system limitations (e.g., "I can't answer because I lack real-time data"). We find that LLM-as-a-Judge systems evaluate ethical refusals significantly more favorably than human users, a divergence not observed for technical refusals. We refer to this divergence as a moderation bias-a systematic tendency for model-based evaluators to reward refusal behaviors more than human users do. This raises broader questions about transparency, value alignment, and the normative assumptions embedded in automated evaluation systems.