# Latent Mechanisms of Language Control in Multilingual Language Models

arXiv: https://arxiv.org/abs/2609.00325 · HTML: https://mlsanity.com/paper/2609.00325/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 10 references that are in the map.
- First posted on arXiv 31 Aug 2026.

## Builds on (most cited first, 3 of 3)

- [Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2](https://mlsanity.com/paper/2408.05147/) (2024)
- [Understanding and Mitigating Language Confusion in LLMs](https://mlsanity.com/paper/2406.20052/) (2024)
- [Language steering in latent space to mitigate unintended code-switching](https://mlsanity.com/paper/2510.13849/) (2025)

## Abstract (arXiv)

Multilingual large language models can exhibit unintended code-switching -- unnecessarily alternating between languages during generation. We present a comparative study of three methods that identify language-controlling latents in cross-layer transcoders: activation value-based selection (ValSel), activation frequency-based selection (FreqSel), and LLM-generated latent annotation-based selection (AnnSel). To evaluate the efficacy of these methods in identifying language-controlling latents, we introduce two multilingual benchmarks that exhibit code-switching for fine-grained analysis of language steering across seven languages. Through targeted intervention experiments on Gemma-2-2B and Qwen3-4B, we find that all three methods effectively manipulate generation language, with FreqSel achieving the strongest overall performance, while AnnSel offering interpretable latent selection through explicit language annotations. A knock-out analysis suggests the methods select non-overlapping but each-functional latent subsets, indicating redundancy rather than a single canonical language direction. Code and data can be found at https://github.com/rm-3284/Latent-Mechanism-Multilingual.