# Can Dialects Be Steered Like Languages? Sparse Neurons and Distributed Directions in Arabic LLMs

arXiv: https://arxiv.org/abs/2607.03936 · HTML: https://mlsanity.com/paper/2607.03936/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 30 references that are in the map.
- First posted on arXiv 4 Jul 2026.

## Builds on (most cited first, 17 of 17)

- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [Persona Vectors: Monitoring and Controlling Character Traits in Language Models](https://mlsanity.com/paper/2507.21509/) (2025)
- [Analyzing Individual Neurons in Pre-trained Language Models](https://mlsanity.com/paper/2010.02695/) (2020)
- [Fanar: An Arabic-Centric Multimodal Generative AI Platform](https://mlsanity.com/paper/2501.13944/) (2025)
- [ALLaM: Large Language Models for Arabic and English](https://mlsanity.com/paper/2407.15390/) (2024)
- [AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMs](https://mlsanity.com/paper/2409.11404/) (2024)
- [NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Task](https://mlsanity.com/paper/2310.16117/) (2023)
- [Discovering Salient Neurons in Deep NLP Models](https://mlsanity.com/paper/2206.13288/) (2022)
- [On the Transformation of Latent Space in Fine-Tuned NLP Models](https://mlsanity.com/paper/2210.12696/) (2022)
- [Language Arithmetics: Towards Systematic Language Neuron Identification and Manipulation](https://mlsanity.com/paper/2507.22608/) (2025)
- [AL-QASIDA: Analyzing LLM Quality and Accuracy Systematically in Dialectal Arabic](https://mlsanity.com/paper/2412.04193/) (2024)
- [Unveiling the Influence of Amplifying Language-Specific Neurons](https://mlsanity.com/paper/2507.22581/) (2025)
- [When Alignment Hurts: Decoupling Representational Spaces in Multilingual Models](https://mlsanity.com/paper/2508.12803/) (2025)
- [UI-Level Evaluation of ALLaM 34B: Measuring an Arabic-Centric LLM via HUMAIN Chat](https://mlsanity.com/paper/2508.17378/) (2025)
- [Post-hoc analysis of Arabic transformer models](https://mlsanity.com/paper/2210.09990/) (2022)
- [From FusHa to Folk: Exploring Cross-Lingual Transfer in Arabic Language Models](https://mlsanity.com/paper/2602.09826/) (2026)

## Abstract (arXiv)

Dialectal data are scarce relative to Modern Standard Arabic (MSA), causing Arabic LLMs to overproduce MSA and struggle with dialectally accurate generation. This raises a fundamental interpretability question about where and how dialectal features are encoded within model internals and whether these representations can improve dialect generation without fine-tuning. We study two inference-time approaches as interpretability probes and control mechanisms. First, neuron-level analysis identifies sparse populations that encode dialect-specific features and tests whether amplifying or suppressing them steers model outputs toward target dialects. Second, vector steering extracts dialect-specific activation directions and injects them during inference, motivated by feature entanglement at the neuron level. We find that these neurons are real but only partially explanatory. They occupy under 1% of MLP dimensions but span only 5% to 21% of the residual dialect direction. This limited coverage is causally consequential. Neuron steering reinforces dialect in some varieties when the prompt is already dialectal but cannot induce it from MSA prompts, whereas vector steering succeeds in both settings. Arabic dialects are therefore steerable mainly through distributed rather than localized representations