# Safer or Luckier? LLMs as Safety Evaluators Are Not Robust to Artifacts

arXiv: https://arxiv.org/abs/2503.09347 · HTML: https://mlsanity.com/paper/2503.09347/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 37 references that are in the map.
- First posted on arXiv 12 Mar 2025.

## Builds on (most cited first, 17 of 17)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models](https://mlsanity.com/paper/2404.18796/) (2024)
- [Humans or LLMs as the Judge? A Study on Judgement Biases](https://mlsanity.com/paper/2402.10669/) (2024)
- [Evaluating Large Language Models at Evaluating Instruction Following](https://mlsanity.com/paper/2310.07641/) (2023)
- [Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges](https://mlsanity.com/paper/2406.12624/) (2024)
- [Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment](https://mlsanity.com/paper/2402.14016/) (2024)
- [PRD: Peer Rank and Discussion Improve Large Language Model based Evaluations](https://mlsanity.com/paper/2307.02762/) (2023)
- [AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policies](https://mlsanity.com/paper/2407.17436/) (2024)
- [LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Models](https://mlsanity.com/paper/2307.07889/) (2023)
- [The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harm](https://mlsanity.com/paper/2406.18682/) (2024)
- [Large Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learning](https://mlsanity.com/paper/2305.17256/) (2023)
- [The price of debiasing automatic metrics in natural language evaluation](https://mlsanity.com/paper/1807.02202/) (2018)
- [The Challenges of Evaluating LLM Applications: An Analysis of Automated, Human, and LLM-Based Approaches](https://mlsanity.com/paper/2406.03339/) (2024)
- [Shortcut Learning in In-Context Learning: A Survey](https://mlsanity.com/paper/2411.02018/) (2024)

## Built on by (most cited first, 26 of 26)

- [Neither Valid nor Reliable? Investigating the Use of LLMs as Judges](https://mlsanity.com/paper/2508.18076/) (2025)
- [The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs](https://mlsanity.com/paper/2506.11094/) (2025)
- [Multi-Agent Debate for LLM Judges with Adaptive Stability Detection](https://mlsanity.com/paper/2510.12697/) (2025)
- [A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustness](https://mlsanity.com/paper/2603.06594/) (2026)
- [Adaptive Defense against Harmful Fine-Tuning for Large Language Models via Bayesian Data Scheduler](https://mlsanity.com/paper/2510.27172/) (2025)
- [A Good Plan is Hard to Find: Aligning Models with Preferences is Misaligned with What Helps Users](https://mlsanity.com/paper/2509.18632/) (2025)
- [Response-Based Knowledge Distillation for Multilingual Jailbreak Prevention Unwittingly Compromises Safety](https://mlsanity.com/paper/2602.11157/) (2025)
- [IndicSafe: A Benchmark for Evaluating Multilingual LLM Safety in South Asia](https://mlsanity.com/paper/2603.17915/) (2026)
- [Safety is Contextual, LLM-Judges Are Not: Navigating the Rigid Priors of Evaluators](https://mlsanity.com/paper/2606.07874/) (2026)
- [Hail to the Thief: Exploring Attacks and Defenses in Decentralised GRPO](https://mlsanity.com/paper/2511.09780/) (2025)
- [How Long Reasoning Chains Influence LLMs' Judgment of Answer Factuality](https://mlsanity.com/paper/2604.06756/) (2026)
- [Structuring the Space of Sociotechnical Alignment](https://mlsanity.com/paper/2607.01250/) (2026)
- [Simple Role Assignment is Extraordinarily Effective for Safety Alignment](https://mlsanity.com/paper/2602.00061/) (2026)
- [When Scanners Lie: Evaluator Instability in LLM Red-Teaming](https://mlsanity.com/paper/2603.14633/) (2026)
- [LLM Judges Inconsistently Disagree Across Safety Criteria and Harm Categories](https://mlsanity.com/paper/2605.31381/) (2026)
- [Empathy Is Not What Changed: Clinical Assessment of Psychological Safety Across GPT Model Generations](https://mlsanity.com/paper/2603.09997/) (2026)
- [Why Do Safety Guardrails Degrade Across Languages?](https://mlsanity.com/paper/2605.17173/) (2026)
- [Safety Measurements for Fine-tuned LLMs Should be Grounded in Capability](https://mlsanity.com/paper/2606.03648/) (2026)
- [ReCal3R: Reliability-Calibrated Learning Rates for Streaming 3D Reconstruction](https://mlsanity.com/paper/2607.05356/) (2026)
- [SAGE: Safety-First Defense-in-Depth Guardrails for Verified Lifecycle Control of High-Impact Generative AI](https://mlsanity.com/paper/2607.22926/) (2026)
- [aipsy-judge: A Specialized, Psychologist-Corrected Local Judge for the Psychological Safety of Conversational AI](https://mlsanity.com/paper/2608.24899/) (2026)
- [Symbolic Temporal Supervision of LLM Agents Using Contracts](https://mlsanity.com/paper/2609.18128/) (2026)
- [StepCOPS: Closed-Testing Lower-Tail Certificates for Language-Model Policy Selection](https://mlsanity.com/paper/2609.29549/) (2026)
- [Pinned and Still Unstable: Within-Judge Verdict Variance and the Noise Floor of LLM-as-Judge Leaderboards](https://mlsanity.com/paper/2609.33044/) (2026)
- [RAIM: Robust Aggregation of Inexpensive Models for Hallucination Detection](https://mlsanity.com/paper/2609.39229/) (2026)
- [Does Target Alignment Mean Target Recovery? An Evidence-Ladder Study of Adversarial Claims on Contrastive Encoders](https://mlsanity.com/paper/2610.11938/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly employed as automated evaluators to assess the safety of generated content, yet their reliability in this role remains uncertain. This study evaluates a diverse set of 11 LLM judge models across critical safety domains, examining three key aspects: self-consistency in repeated judging tasks, alignment with human judgments, and susceptibility to input artifacts such as apologetic or verbose phrasing. Our findings reveal that biases in LLM judges can significantly distort the final verdict on which content source is safer, undermining the validity of comparative evaluations. Notably, apologetic language artifacts alone can skew evaluator preferences by up to 98%. Contrary to expectations, larger models do not consistently exhibit greater robustness, while smaller models sometimes show higher resistance to specific artifacts. To mitigate LLM evaluator robustness issues, we investigate jury-based evaluations aggregating decisions from multiple models. Although this approach both improves robustness and enhances alignment to human judgements, artifact sensitivity persists even with the best jury configurations. These results highlight the urgent need for diversified, artifact-resistant methodologies to ensure reliable safety assessments.