Safety Is Not Universal: The Selective Safety Trap in LLM Alignment
On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks
also in 1 more
Abstract (arXiv)
Current safety evaluations of large language models (LLMs) create a dangerous illusion of universal protection by aggregating harms under generic categories such as "Identity Hate", obscuring vulnerabilities toward specific populations. In this work, we expose the Selective Safety Trap: a systemic failure mode where models robustly defend specific populations while leaving underrepresented communities highly vulnerable to identical adversarial attacks. To systematically audit this phenomenon, we introduce MiJaBench, a bilingual (English-Portuguese) adversarial benchmark comprising 43,961 controlled jailbreaking prompts across 16 minority groups. By evaluating 14 state-of-the-art LLMs on MiJaBench, we curate 615,454 prompt-response pairs that compose MiJaBench-Align, revealing that safety alignment is not a uniform semantic capability but a demographic hierarchy, with defense rates fluctuating by up to 42% within the same model solely based on the target group. This disparity persists across architectures and languages and is amplified by scaling, indicating that current alignment methods learn group-specific safeguards rather than a generalized notion of harm. Through targeted direct preference optimization (DPO) on a 1B-parameter baseline, we achieve strong zero-shot safety generalizations to entirely unseen demographics and complex attack strategies. We release all datasets and scripts to provide the community with a concrete pathway toward equitable, transferable safety alignment.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 19 of its 44 references that are in the map. First posted on arXiv 7 Jan 2026.
Builds on
19 of 44 references.
Closest ancestors · 2
Same direction, newest first
- 2025Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLM7 Aug
- 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset10 Jul
Other references · 17
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
- 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
- 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
- 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
- 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 496
- 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
- 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 422
- 2023Low-Resource Languages Jailbreak GPT-4cited by 309
- 2025Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrailscited by 132
5 more
- 2024Echoes in AI: Quantifying lack of plot diversity in LLM outputscited by 48
- 2025Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judgecited by 23
- 2025Towards Safer Pretraining: Analyzing and Filtering Harmful Content in Webscale datasets for Responsible LLMscited by 9
- 2025Guarding the Guardrails: A Taxonomy-Driven Approach to Jailbreak Detectioncited by 5
- 2025ToxSyn-PT: A Synthetic Fine-Grained Dataset of Minority-Targeted Toxic Language in Portuguesecited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deep
- 2026Understanding the Effects of Safety Unalignment on Large Language Models
- 2024Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models
- 2025Safety Alignment Can Be Not Superficial With Explicit Safety Signals
- 2025Targeting Alignment: Extracting Safety Classifiers of Aligned LLMs
- 2025SABER: Uncovering Vulnerabilities in Safety Alignment via Cross-Layer Residual Connection
- 2025Safety Alignment Should Be Made More Than Just A Few Attention Heads
- 2025Playing the Fool: Jailbreaking LLMs and Multimodal LLMs with Out-of-Distribution Strategy