# ReCoVeR the Target Language: Language Steering without Sacrificing Task Performance

arXiv: https://arxiv.org/abs/2509.14814 · HTML: https://mlsanity.com/paper/2509.14814/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 30 references that are in the map.
- First posted on arXiv 18 Sep 2025.

## Builds on (most cited first, 21 of 21)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering](https://mlsanity.com/paper/2311.06668/) (2023)
- [CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task](https://mlsanity.com/paper/2209.06243/) (2022)
- [Improving Instruction-Following in Language Models through Activation Steering](https://mlsanity.com/paper/2410.12877/) (2024)
- [WikiLingua: A New Benchmark Dataset for Cross-Lingual Abstractive Summarization](https://mlsanity.com/paper/2010.03093/) (2020)
- [How do Large Language Models Handle Multilingualism?](https://mlsanity.com/paper/2402.18815/) (2024)
- [Inspecting and Editing Knowledge Representations in Language Models](https://mlsanity.com/paper/2304.00740/) (2023)
- [Aya 23: Open Weight Releases to Further Multilingual Progress](https://mlsanity.com/paper/2405.15032/) (2024)
- [Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization](https://mlsanity.com/paper/2406.00045/) (2024)
- [On the Multilingual Ability of Decoder-based Pre-trained Language Models: Finding and Controlling Language-Specific Neurons](https://mlsanity.com/paper/2404.02431/) (2024)
- [Understanding and Mitigating Language Confusion in LLMs](https://mlsanity.com/paper/2406.20052/) (2024)
- [Improving Activation Steering in Language Models with Mean-Centring](https://mlsanity.com/paper/2312.03813/) (2023)
- [Semantics-Adaptive Activation Intervention for LLMs via Dynamic Steering Vectors](https://mlsanity.com/paper/2410.12299/) (2024)
- [Representation Surgery: Theory and Practice of Affine Steering](https://mlsanity.com/paper/2402.09631/) (2024)
- [CrossSum: Beyond English-Centric Cross-Lingual Summarization for 1,500+ Language Pairs](https://mlsanity.com/paper/2112.08804/) (2021)
- [Activation Scaling for Steering and Interpreting Language Models](https://mlsanity.com/paper/2410.04962/) (2024)
- [Evaluating the Elementary Multilingual Capabilities of Large Language Models with MultiQ](https://mlsanity.com/paper/2403.03814/) (2024)
- [Unveiling Multilinguality in Transformer Models: Exploring Language Specificity in Feed-Forward Networks](https://mlsanity.com/paper/2310.15552/) (2023)

## Built on by (most cited first, 6 of 6)

- [Compositional Steering of Large Language Models with Steering Tokens](https://mlsanity.com/paper/2601.05062/) (2026)
- [Language Steering for Multilingual In-Context Learning](https://mlsanity.com/paper/2602.02326/) (2026)
- [SteerEval: Inference-time Interventions Strengthen Multilingual Generalization in Neural Summarization Metrics](https://mlsanity.com/paper/2601.15809/) (2026)
- [Strengthening Target-Language Features: SAE-Based Steering for Multilingual Inference](https://mlsanity.com/paper/2608.04904/) (2026)
- [Target-Language Generation in Multilingual Models: Activation Steering and Optimal Control](https://mlsanity.com/paper/2609.16967/) (2026)
- [Language Model Activations Inhabit Privileged Error-Correcting Basins](https://mlsanity.com/paper/2610.04183/) (2026)

## Abstract (arXiv)

As they become increasingly multilingual, Large Language Models (LLMs) exhibit more language confusion, i.e., they tend to generate answers in a language different from the language of the prompt or the answer language explicitly requested by the user. In this work, we propose ReCoVeR (REducing language COnfusion in VEctor Representations), a novel lightweight approach for reducing language confusion based on language-specific steering vectors. We first isolate language vectors with the help of multi-parallel corpus and then effectively leverage those vectors for effective LLM steering via fixed (i.e., unsupervised) as well as trainable steering functions. Our extensive evaluation, encompassing three benchmarks and 18 languages, shows that ReCoVeR effectively mitigates language confusion in both monolingual and cross-lingual setups while at the same time -- and in contrast to prior language steering methods -- retaining task performance. Our data code is available at https://github.com/hSterz/recover.