# Cultural Binding Heads in Language Models

arXiv: https://arxiv.org/abs/2605.28543 · HTML: https://mlsanity.com/paper/2605.28543/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 23 references that are in the map.
- First posted on arXiv 27 May 2026.

## Builds on (most cited first, 15 of 15)

- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [The Hydra Effect: Emergent Self-repair in Language Model Computations](https://mlsanity.com/paper/2307.15771/) (2023)
- [How do Language Models Bind Entities in Context?](https://mlsanity.com/paper/2310.17191/) (2023)
- [Discovering Variable Binding Circuitry with Desiderata](https://mlsanity.com/paper/2307.03637/) (2023)
- [Query-Focused Retrieval Heads Improve Long-Context Reasoning and Re-ranking](https://mlsanity.com/paper/2506.09944/) (2025)
- [Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMs](https://mlsanity.com/paper/2502.01926/) (2025)
- [Localized Cultural Knowledge is Conserved and Controllable in Large Language Models](https://mlsanity.com/paper/2504.10191/) (2025)
- [Isolating Culture Neurons in Multilingual Large Language Models](https://mlsanity.com/paper/2508.02241/) (2025)
- [Steering LLMs for Culturally Localized Generation](https://mlsanity.com/paper/2603.23301/) (2026)
- [Finding Culture-Sensitive Neurons in Vision-Language Models](https://mlsanity.com/paper/2510.24942/) (2025)
- [BiasGym: A Simple and Generalizable Framework for Analyzing and Removing Biases through Injection](https://mlsanity.com/paper/2508.08855/) (2025)
- [Neuron-Level Analysis of Cultural Understanding in Large Language Models](https://mlsanity.com/paper/2510.08284/) (2025)

## Abstract (arXiv)

LLMs often default to equal treatment across cultural groups, even though context warrants differentiation: this is a lack of difference awareness. Using mechanistic interpretability and a factorial design on the N4 cultural appropriation benchmark from Wang et al. (2025), we identify 2-3 mid-layer attention heads per model that contribute causally to cultural binding across eight models (base and instruct versions of four architectures). Cultural binding is the process of associating a cultural item with its related identity. Knockout of the identity-to-item edges on these heads lowers the binding strength by 9-23%. The identified heads transfer from instruct to base models, suggesting that cultural binding is created during pre-training. An α-scaling shows a graded dose-response. Moderate amplification steering at generation (α = 2-3) increases cultural differentiation accuracy by 1-3 pp while leaving reasoning on culturally neutral questions mostly intact. A knowledge probing task shows that models know 3-6 times more than they act upon, indicating that the bottleneck lies in routing and not knowledge.