mlsanityData to 14 Oct 2026

Cross-Lingual Activation Steering for Multilingual Language Models

arXiv 2601.16390 ↗First posted 23 Jan 2026cs.CL cs.AIcited by 5

On the map: Mechanistic interpretability → Activation steering methods

Abstract (arXiv)

Large language models exhibit strong multilingual capabilities, yet significant performance gaps persist between dominant and non-dominant languages. Prior work attributes this gap to imbalances between shared and language-specific neurons in multilingual representations. We propose Cross-Lingual Activation Steering (CLAS), a training-free inference-time intervention that selectively modulates neuron activations. We evaluate CLAS on classification and generation benchmarks, achieving average improvements of 2.3% (Acc.) and 3.4% (F1) respectively, while maintaining high-resource language performance. We discover that effective transfer operates through functional divergence rather than strict alignment; performance gains correlate with increased language cluster separation. Our results demonstrate that targeted activation steering can unlock latent multilingual capacity in existing models without modification to model weights.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability. Builds on 24 of its 35 references that are in the map. First posted on arXiv 23 Jan 2026.

Builds on

24 of 35 references.

References · 24

  1. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  2. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 578
  3. 2019PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identificationcited by 264
  4. 2024How do Large Language Models Handle Multilingualism?cited by 150
  5. 2023Do Multilingual Language Models Think Better in English?cited by 88
  6. 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Biascited by 84
  7. 2024On the Multilingual Ability of Decoder-based Pre-trained Language Models: Finding and Controlling Language-Specific Neuronscited by 83
  8. 2024Multilingual Instruction Tuning With Just a Pinch of Multilingualitycited by 73
  9. 2023Multilingual LLMs are Better Cross-lingual In-context Learners with Alignmentcited by 71
  10. 2024MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimizationcited by 69
  11. 2023Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpacacited by 60
  12. 2024LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedbackcited by 51
12 more
  1. 2024Enhancing Multilingual Capabilities of Large Language Models through Self-Distillation from Resource-Rich Languagescited by 36
  2. 2024AdaMergeX: Cross-Lingual Transfer with Large Language Models via Adaptive Adapter Mergingcited by 34
  3. 2024Sharing Matters: Analysing Neurons Across Languages and Tasks in LLMscited by 33
  4. 2024Multilingual Pretraining and Instruction Tuning Improve Cross-Lingual Knowledge Alignment, But Only Shallowlycited by 27
  5. 2025Language-Specific Latent Process Hinders Cross-Lingual Performancecited by 21
  6. 2024Large Language Models Are Cross-Lingual Knowledge-Free Reasonerscited by 16
  7. 2025Steering into New Embedding Spaces: Analyzing Cross-Lingual Alignment Induced by Model Interventions in Multilingual Language Modelscited by 11
  8. 2025Language-specific Neurons Do Not Facilitate Cross-Lingual Transfercited by 11
  9. 2025From Neurons to Semantics: Evaluating Cross-Linguistic Alignment Capabilities of Large Language Models via Neurons Alignmentcited by 9
  10. 2025How Does Alignment Enhance LLMs' Multilingual Capabilities? A Language Neurons Perspectivecited by 8
  11. 2025The Transfer Neurons Hypothesis: An Underlying Mechanism for Language Latent Space Transitions in Multilingual LLMscited by 8
  12. 2025Enhancing LLM Language Adaption through Cross-lingual In-Context Pre-trainingcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Language Steering for Multilingual In-Context Learningcited by 3
  2. 2026Soft Token Alignment for Cross-Lingual Reasoningcited by 0
  3. 2026Cross-lingual Functional Vectors for Emotion Detection in Large Language Modelscited by 0
  4. 2026Choosing the Right Language Mode at Inference Time for Multilingual Reliabilitycited by 0
  5. 2026Retrieval Disagreement as a Signal: Diagnosing and Steering Multilingual Dense Embedderscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026CLaS-Bench: A Cross-Lingual Alignment and Steering Benchmark
  2. 2024Converging to a Lingua Franca: Evolution of Linguistic Regions and Semantics Alignment in Multilingual Large Language Models
  3. 2025Enhancing Cross-task Transfer of Large Language Models via Activation Steering
  4. 2026Multilingual Steering by Design: Multilingual Sparse Autoencoders and Principled Layer Selection
  5. 2024Probing the Emergence of Cross-lingual Alignment during LLM Training
  6. 2026Contextual Linear Activation Steering of Language Models
  7. 2025Improving Multilingual Language Models by Aligning Representations through Steering
  8. 2024Activation Scaling for Steering and Interpreting Language Models