# Breaking Down Bias: On The Limits of Generalizable Pruning Strategies

arXiv: https://arxiv.org/abs/2502.07771 · HTML: https://mlsanity.com/paper/2502.07771/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 69 references that are in the map.
- First posted on arXiv 11 Feb 2025.

## Builds on (most cited first, 25 of 25)

- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned](https://mlsanity.com/paper/1905.09418/) (2019)
- [SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning](https://mlsanity.com/paper/2012.09852/) (2020)
- [Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications](https://mlsanity.com/paper/2402.05162/) (2024)
- [Gender bias and stereotypes in Large Language Models](https://mlsanity.com/paper/2308.14921/) (2023)
- [Large Language Models are Geographically Biased](https://mlsanity.com/paper/2402.02680/) (2024)
- [Fairness And Bias in Artificial Intelligence: A Brief Survey of Sources, Impacts, And Mitigation Strategies](https://mlsanity.com/paper/2304.07683/) (2023)
- [AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policies](https://mlsanity.com/paper/2407.17436/) (2024)
- [Clio: Privacy-Preserving Insights into Real-World AI Use](https://mlsanity.com/paper/2412.13678/) (2024)
- [Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions](https://mlsanity.com/paper/2410.02584/) (2024)
- [The Unequal Opportunities of Large Language Models: Revealing Demographic Bias through Job Recommendations](https://mlsanity.com/paper/2308.02053/) (2023)
- [AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies](https://mlsanity.com/paper/2406.17864/) (2024)
- [Do Localization Methods Actually Localize Memorized Data in LLMs? A Tale of Two Benchmarks](https://mlsanity.com/paper/2311.09060/) (2023)
- [Mitigating Biases for Instruction-following Language Models via Bias Neurons Elimination](https://mlsanity.com/paper/2311.09627/) (2023)
- [Fairness-Aware Structured Pruning in Transformers](https://mlsanity.com/paper/2312.15398/) (2023)
- [An overview of domain-specific foundation model: key technologies, applications and challenges](https://mlsanity.com/paper/2409.04267/) (2024)
- [Fine-Tuning Games: Bargaining and Adaptation for General-Purpose Models](https://mlsanity.com/paper/2308.04399/) (2023)
- [Know What You Don't Need: Single-Shot Meta-Pruning for Attention Heads](https://mlsanity.com/paper/2011.03770/) (2020)
- [Taking the Next Step with Generative Artificial Intelligence: The Transformative Role of Multimodal Large Language Models in Science Education](https://mlsanity.com/paper/2401.00832/) (2024)
- [From Melting Pots to Misrepresentations: Exploring Harms in Generative AI](https://mlsanity.com/paper/2403.10776/) (2024)
- [Memorized Images in Diffusion Models share a Subspace that can be Located and Deleted](https://mlsanity.com/paper/2406.18566/) (2024)
- [The European AI Liability Directives -- Critique of a Half-Hearted Approach and Lessons for the Future](https://mlsanity.com/paper/2211.13960/) (2022)
- [Acceptable Use Policies for Foundation Models](https://mlsanity.com/paper/2409.09041/) (2024)
- [Attention Speaks Volumes: Localizing and Mitigating Bias in Language Models](https://mlsanity.com/paper/2410.22517/) (2024)
- [Towards Effective Discrimination Testing for Generative AI](https://mlsanity.com/paper/2412.21052/) (2024)

## Built on by (most cited first, 2 of 2)

- [Simulating Teams with LLM Agents: Interactive 2D Environments for Studying Human-AI Dynamics](https://mlsanity.com/paper/2510.08242/) (2025)
- [What AI Speaks for Your Community: Polling AI Agents for Public Opinion on Data Center Projects](https://mlsanity.com/paper/2511.22037/) (2025)

## Abstract (arXiv)

We employ model pruning to examine how LLMs conceptualize racial biases, and whether a generalizable mitigation strategy for such biases appears feasible. Our analysis yields several novel insights. We find that pruning can be an effective method to reduce bias without significantly increasing anomalous model behavior. Neuron-based pruning strategies generally yield better results than approaches pruning entire attention heads. However, our results also show that the effectiveness of either approach quickly deteriorates as pruning strategies become more generalized. For instance, a model that is trained on removing racial biases in the context of financial decision-making poorly generalizes to biases in commercial transactions. Overall, our analysis suggests that racial biases are only partially represented as a general concept within language models. The other part of these biases is highly context-specific, suggesting that generalizable mitigation strategies may be of limited effectiveness. Our findings have important implications for legal frameworks surrounding AI. In particular, they suggest that an effective mitigation strategy should include the allocation of legal responsibility on those that deploy models in a specific use case.