# Intra-Fairness Dynamics: The Bias Spillover Effect in Targeted LLM Alignment

arXiv: https://arxiv.org/abs/2602.16438 · HTML: https://mlsanity.com/paper/2602.16438/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 20 of its 40 references that are in the map.
- First posted on arXiv 18 Feb 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 20 of 20)

- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [Zephyr: Direct Distillation of LM Alignment](https://mlsanity.com/paper/2310.16944/) (2023)
- [Generative Language Models Exhibit Social Identity Biases](https://mlsanity.com/paper/2310.15819/) (2023)
- [DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily Life](https://mlsanity.com/paper/2410.02683/) (2024)
- [A Theoretical Understanding of Self-Correction through In-context Alignment](https://mlsanity.com/paper/2405.18634/) (2024)
- [One Epoch Is All You Need](https://mlsanity.com/paper/1906.06669/) (2019)
- [Technical Challenges for Training Fair Neural Networks](https://mlsanity.com/paper/2102.06764/) (2021)
- [ValueCompass: A Framework for Measuring Contextual Value Alignment Between Human and LLMs](https://mlsanity.com/paper/2409.09586/) (2024)
- [Position: Towards Bidirectional Human-AI Alignment](https://mlsanity.com/paper/2406.09264/) (2024)
- [Advancing LLM Safe Alignment with Safety Representation Ranking](https://mlsanity.com/paper/2505.15710/) (2025)
- [Aligned but Blind: Alignment Increases Implicit Bias by Reducing Awareness of Race](https://mlsanity.com/paper/2506.00253/) (2025)
- [Muffin: A Framework Toward Multi-Dimension AI Fairness by Uniting Off-the-Shelf Models](https://mlsanity.com/paper/2308.13730/) (2023)
- [No Free Lunch in Language Model Bias Mitigation? Targeted Bias Reduction Can Exacerbate Unmitigated LLM Biases](https://mlsanity.com/paper/2511.18635/) (2025)
- [The Tug of War Within: Mitigating the Fairness-Privacy Conflicts in Large Language Models](https://mlsanity.com/paper/2410.16672/) (2024)
- [KLAAD: Refining Attention Mechanisms to Reduce Societal Bias in Generative Language Models](https://mlsanity.com/paper/2507.19962/) (2025)
- [Do LLMs Align Human Values Regarding Social Biases? Judging and Explaining Social Biases with LLMs](https://mlsanity.com/paper/2509.13869/) (2025)
- [Enhancing Multi-Attribute Fairness in Healthcare Predictive Modeling](https://mlsanity.com/paper/2501.13219/) (2025)

## Built on by (most cited first, 1 of 1)

- [Unfair and Unsafe: Everyday Bias Triggers Emergent Misalignment](https://mlsanity.com/paper/2610.15961/) (2026)

## Abstract (arXiv)

Conventional large language model (LLM) fairness alignment largely focuses on mitigating bias along single sensitive attributes, overlooking fairness as an inherently multidimensional and context-specific value. This approach risks creating systems that achieve narrow fairness metrics while exacerbating disparities along untargeted attributes, a phenomenon known as bias spillover. While extensively studied in machine learning, bias spillover remains critically underexplored in LLM alignment. In this work, we investigate how targeted gender alignment affects fairness across nine sensitive attributes in three state-of-the-art LLMs (Mistral 7B, Llama 3.1 8B, Qwen 2.5 7B). Using Direct Preference Optimization and the BBQ benchmark, we evaluate fairness under ambiguous and disambiguous contexts. Our findings reveal noticeable bias spillover: while aggregate results show improvements, context-aware analysis exposes significant degradations in ambiguous contexts, particularly for physical appearance (p< 0.001 across all models), sexual orientation, and disability status. We demonstrate that improving fairness along one attribute can inadvertently worsen disparities in others under uncertainty, highlighting the necessity of context-aware, multi-attribute fairness evaluation frameworks.