# Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judge

arXiv: https://arxiv.org/abs/2504.07887 · HTML: https://mlsanity.com/paper/2504.07887/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security.
- Builds on 50 of its 74 references that are in the map.
- First posted on arXiv 10 Apr 2025.

## Placed in

- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)

## Builds on (most cited first, 30 of 50)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models](https://mlsanity.com/paper/2310.04451/) (2023)
- [Phi-4 Technical Report](https://mlsanity.com/paper/2412.08905/) (2024)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [Tree of Attacks: Jailbreaking Black-Box LLMs Automatically](https://mlsanity.com/paper/2312.02119/) (2023)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [Prometheus: Inducing Fine-grained Evaluation Capability in Language Models](https://mlsanity.com/paper/2310.08491/) (2023)
- [BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation](https://mlsanity.com/paper/2101.11718/) (2021)
- [On Measuring Social Biases in Sentence Encoders](https://mlsanity.com/paper/1903.10561/) (2019)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [JudgeLM: Fine-tuned Large Language Models are Scalable Judges](https://mlsanity.com/paper/2310.17631/) (2023)
- [Low-Resource Languages Jailbreak GPT-4](https://mlsanity.com/paper/2310.02446/) (2023)
- [Measuring Bias in Contextualized Word Representations](https://mlsanity.com/paper/1906.07337/) (2019)
- [Self-Preference Bias in LLM-as-a-Judge](https://mlsanity.com/paper/2410.21819/) (2024)
- [Jailbreak Attacks and Defenses Against Large Language Models: A Survey](https://mlsanity.com/paper/2407.04295/) (2024)
- [Identifying and Reducing Gender Bias in Word-Level Language Models](https://mlsanity.com/paper/1904.03035/) (2019)
- [On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective](https://mlsanity.com/paper/2302.12095/) (2023)
- [Gender bias and stereotypes in Large Language Models](https://mlsanity.com/paper/2308.14921/) (2023)
- [Generative Judge for Evaluating Alignment](https://mlsanity.com/paper/2310.05470/) (2023)

## Built on by (most cited first, 23 of 23)

- [A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models](https://mlsanity.com/paper/2509.03871/) (2025)
- [Oyster-I: Beyond Refusal -- Constructive Safety Alignment for Responsible Language Models](https://mlsanity.com/paper/2509.01909/) (2025)
- [Don't Judge Code by Its Cover: Exploring Biases in LLM Judges for Code Evaluation](https://mlsanity.com/paper/2505.16222/) (2025)
- [Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metrics](https://mlsanity.com/paper/2505.18658/) (2025)
- [Fooling the LVLM Judges: Visual Biases in LVLM-Based Evaluation](https://mlsanity.com/paper/2505.15249/) (2025)
- [Can You Trick the Grader? Adversarial Persuasion of LLM Judges](https://mlsanity.com/paper/2508.07805/) (2025)
- [Efficient Switchable Safety Control in LLMs via Magic-Token-Guided Co-Training](https://mlsanity.com/paper/2508.14904/) (2025)
- [Benchmarking Educational LLMs with Analytics: A Case Study on Gender Bias in Feedback](https://mlsanity.com/paper/2511.08225/) (2025)
- [Chain-of-Thought Driven Adversarial Scenario Extrapolation for Robust Language Models](https://mlsanity.com/paper/2505.17089/) (2025)
- [Is Reasoning All You Need? Probing Bias in the Age of Reasoning Language Models](https://mlsanity.com/paper/2507.02799/) (2025)
- [Measuring and Exploiting Contextual Bias in LLM-Assisted Security Code Review](https://mlsanity.com/paper/2603.18740/) (2026)
- [When LLMs Imagine People: A Human-Centered Persona Brainstorm Audit for Bias and Fairness in Creative Applications](https://mlsanity.com/paper/2602.00044/) (2026)
- [MetaCrit: A Critical Thinking Framework for Self-Regulated LLM Reasoning](https://mlsanity.com/paper/2507.15015/) (2025)
- [Fairness Testing in Retrieval-Augmented Generation: How Small Perturbations Reveal Bias in Small Language Models](https://mlsanity.com/paper/2509.26584/) (2025)
- [Toward Reliable, Safe, and Secure LLMs for Scientific Applications](https://mlsanity.com/paper/2603.18235/) (2026)
- [BiAxisBias: Evaluating LLM Bias Beyond a Single Prompt and a Single Explanation](https://mlsanity.com/paper/2605.09041/) (2026)
- [Fair outputs, Biased Internals: Causal Potency and Asymmetry of Latent Bias in LLMs for High-Stakes Decisions](https://mlsanity.com/paper/2605.15217/) (2026)
- [Safety Is Not Universal: The Selective Safety Trap in LLM Alignment](https://mlsanity.com/paper/2601.04389/) (2026)
- [Making Bias Non-Predictive: Training Robust LLM Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2602.01528/) (2026)
- [Evaluating Second-Order Bias of LLMs Through Epistemic Entitlement](https://mlsanity.com/paper/2606.17506/) (2026)
- [Exposing the Unsaid: Visualizing Hidden LLM Bias through Stochastic Path Aggregation](https://mlsanity.com/paper/2606.19344/) (2026)
- [LLM-Driven Autonomous Vehicles Inherit Human Driver Biases in Pedestrian Yielding: Results and Implications From A New Benchmark](https://mlsanity.com/paper/2609.00192/) (2026)
- [When Noise Fabricates Bias: The Fragility of LLM-as-a-Judge Bias Measurement under Noisy Text](https://mlsanity.com/paper/2609.11067/) (2026)

## Abstract (arXiv)

The growing integration of Large Language Models (LLMs) into critical societal domains has raised concerns about embedded biases that can perpetuate stereotypes and undermine fairness. Such biases may stem from historical inequalities in training data, linguistic imbalances, or adversarial manipulation. Despite mitigation efforts, recent studies show that LLMs remain vulnerable to adversarial attacks that elicit biased outputs. This work proposes a scalable benchmarking framework to assess LLM robustness to adversarial bias elicitation. Our methodology involves: (i) systematically probing models across multiple tasks targeting diverse sociocultural biases, (ii) quantifying robustness through safety scores using an LLM-as-a-Judge approach, and (iii) employing jailbreak techniques to reveal safety vulnerabilities. To facilitate systematic benchmarking, we release a curated dataset of bias-related prompts, named CLEAR-Bias. Our analysis, identifying DeepSeek V3 as the most reliable judge LLM, reveals that bias resilience is uneven, with age, disability, and intersectional biases among the most prominent. Some small models outperform larger ones in safety, suggesting that training and architecture may matter more than scale. However, no model is fully robust to adversarial elicitation, with jailbreak attacks using low-resource languages or refusal suppression proving effective across model families. We also find that successive LLM generations exhibit slight safety gains, while models fine-tuned for the medical domain tend to be less safe than their general-purpose counterparts.