# Code-Switching Reveals Language Anchoring in Multilingual LLMs

arXiv: https://arxiv.org/abs/2606.19668 · HTML: https://mlsanity.com/paper/2606.19668/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 45 references that are in the map.
- First posted on arXiv 18 Jun 2026.

## Builds on (most cited first, 30 of 30)

- [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization](https://mlsanity.com/paper/2003.11080/) (2020)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [Do Llamas Work in English? On the Latent Language of Multilingual Transformers](https://mlsanity.com/paper/2402.10588/) (2024)
- [XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation](https://mlsanity.com/paper/2004.01401/) (2020)
- [All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality](https://mlsanity.com/paper/2109.04404/) (2021)
- [How do Large Language Models Handle Multilingualism?](https://mlsanity.com/paper/2402.18815/) (2024)
- [Dissecting Contextual Word Embeddings: Architecture and Representation](https://mlsanity.com/paper/1808.08949/) (2018)
- [Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models](https://mlsanity.com/paper/2310.10378/) (2023)
- [A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologies](https://mlsanity.com/paper/2301.01967/) (2023)
- [SimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric Knowledge](https://mlsanity.com/paper/2509.07968/) (2025)
- [Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages](https://mlsanity.com/paper/2303.13592/) (2023)
- [Investigating Language Preference of Multilingual RAG Systems](https://mlsanity.com/paper/2502.11175/) (2025)
- [Language-Specific Latent Process Hinders Cross-Lingual Performance](https://mlsanity.com/paper/2505.13141/) (2025)
- [Language Representation Projection: Can We Transfer Factual Knowledge across Languages in Multilingual Language Models?](https://mlsanity.com/paper/2311.03788/) (2023)
- [Improving Multilingual Language Models by Aligning Representations through Steering](https://mlsanity.com/paper/2505.12584/) (2025)
- [Lost in the Mix: Evaluating LLM Understanding of Code-Switched Text](https://mlsanity.com/paper/2506.14012/) (2025)
- [Understanding Multilingualism in Mixture-of-Experts LLMs: Routing Mechanism, Expert Specialization, and Layerwise Steering](https://mlsanity.com/paper/2601.14050/) (2026)
- [Are Pretrained Multilingual Models Equally Fair Across Languages?](https://mlsanity.com/paper/2210.05457/) (2022)
- [Minimal Pair-Based Evaluation of Code-Switching](https://mlsanity.com/paper/2506.01840/) (2025)
- [Code-Mixed Probes Show How Pre-Trained Models Generalise On Code-Switched Text](https://mlsanity.com/paper/2403.04872/) (2024)
- [Faux Polyglot: A Study on Information Disparity in Multilingual Large Language Models](https://mlsanity.com/paper/2407.05502/) (2024)
- [Can Code-Switched Texts Activate a Knowledge Switch in LLMs? A Case Study on English-Korean Code-Switching](https://mlsanity.com/paper/2410.18436/) (2024)
- [Can Large Language Models Understand, Reason About, and Generate Code-Switched Text?](https://mlsanity.com/paper/2601.07153/) (2026)
- [Unveiling Language Routing Isolation in Multilingual MoE Models for Interpretable Subnetwork Adaptation](https://mlsanity.com/paper/2604.03592/) (2026)
- [Code-Switching and Syntax: A Large-Scale Experiment](https://mlsanity.com/paper/2506.01846/) (2025)
- [OLA: Output Language Alignment in Code-Switched LLM Interactions](https://mlsanity.com/paper/2601.03589/) (2026)
- [Language Steering for Multilingual In-Context Learning](https://mlsanity.com/paper/2602.02326/) (2026)
- [PingPong: A Natural Benchmark for Multi-Turn Code-Switching Dialogues](https://mlsanity.com/paper/2601.17277/) (2026)

## Built on by (most cited first, 1 of 1)

- [Phonological Interference in Multilingual Speech Models](https://mlsanity.com/paper/2610.11275/) (2026)

## Abstract (arXiv)

Multilingual Large Language Models (MLLMs) are increasingly expected to handle Code-Switched (CS) inputs, yet mixing languages frequently degrades performance relative to source- or target-language monolingual counterparts. To understand this degradation, we use grammar-forced CS as a controlled diagnostic setting for locating CS representations relative to their source and target counterparts. We introduce Anchor Bias, a geometric measure that quantifies language anchoring, whether a CS hidden state aligns closer to its source or target language counterpart. Across diverse MLLMs, Anchor Bias reveals a consistent grammar-frame effect: source-framed CS stays source-anchored, whereas target-framed CS shifts target-ward and shows larger Question Answering (QA) degradation. Motivated by this representational pattern, we propose CANVAS (Contextual Anchor-based Neural Vector Alignment Steering), an inference-time intervention that extracts a source-side canvas from the input and softly steers target-language hidden states toward the source anchor during prefill. CANVAS consistently recovers QA F1 across MLLMs and CS conditions, showing that internal anchoring signals provide an actionable target for mitigating CS inference failures.