# Rethinking what Matters: Effective and Robust Multilingual Realignment for Low-Resource Languages

arXiv: https://arxiv.org/abs/2511.06497 · HTML: https://mlsanity.com/paper/2511.06497/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 25 of its 56 references that are in the map.
- First posted on arXiv 9 Nov 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 25 of 25)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [XNLI: Evaluating Cross-lingual Sentence Representations](https://mlsanity.com/paper/1809.05053/) (2018)
- [How multilingual is Multilingual BERT?](https://mlsanity.com/paper/1906.01502/) (2019)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders](https://mlsanity.com/paper/2404.05961/) (2024)
- [Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT](https://mlsanity.com/paper/1904.09077/) (2019)
- [A Survey on Data Selection for Language Models](https://mlsanity.com/paper/2402.16827/) (2024)
- [Word Alignment by Fine-tuning Embeddings on Parallel Corpora](https://mlsanity.com/paper/2101.08231/) (2021)
- [RegMix: Data Mixture as Regression for Language Model Pre-training](https://mlsanity.com/paper/2407.01492/) (2024)
- [A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings](https://mlsanity.com/paper/1805.06297/) (2018)
- [Multilingual Alignment of Contextual Word Representations](https://mlsanity.com/paper/2002.03518/) (2020)
- [Universal Neural Machine Translation for Extremely Low Resource Languages](https://mlsanity.com/paper/1802.05368/) (2018)
- [AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages](https://mlsanity.com/paper/2104.08726/) (2021)
- [Inducing Language-Agnostic Multilingual Representations](https://mlsanity.com/paper/2008.09112/) (2020)
- [IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models](https://mlsanity.com/paper/2406.03368/) (2024)
- [RLHF Can Speak Many Languages: Unlocking Multilingual Preference Optimization for LLMs](https://mlsanity.com/paper/2407.02552/) (2024)
- [Continual Mixed-Language Pre-Training for Extremely Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2105.03953/) (2021)
- [R3: Robust Rubric-Agnostic Reward Models](https://mlsanity.com/paper/2505.13388/) (2025)
- [ProxyLM: Predicting Language Model Performance on Multilingual Tasks via Proxy Models](https://mlsanity.com/paper/2406.09334/) (2024)
- [Exploring the Relationship between Alignment and Cross-lingual Transfer in Multilingual Transformers](https://mlsanity.com/paper/2306.02790/) (2023)
- [Target Conditioned Sampling: Optimizing Data Selection for Multilingual Neural Machine Translation](https://mlsanity.com/paper/1905.08212/) (2019)
- [Myanmar XNLI: Building a Dataset and Exploring Low-resource Approaches to Natural Language Inference with Myanmar](https://mlsanity.com/paper/2504.09645/) (2025)
- [mR3: Multilingual Rubric-Agnostic Reward Reasoning Models](https://mlsanity.com/paper/2510.01146/) (2025)
- [Do Explicit Alignments Robustly Improve Multilingual Encoders?](https://mlsanity.com/paper/2010.02537/) (2020)

## Abstract (arXiv)

Realignment is a promising strategy to improve cross-lingual transfer in multilingual language models. However, empirical results are mixed and often unreliable, particularly for typologically distant or low-resource languages (LRLs) compared to English. Moreover, word realignment tools often rely on high-quality parallel data, which can be scarce or noisy for many LRLs. In this work, we conduct an extensive empirical study to investigate whether realignment truly benefits from using all available languages, or if strategically selected subsets can offer comparable or even improved cross-lingual transfer, and study the impact on LRLs. Our controlled experiments show that realignment can be particularly effective for LRLs and that using carefully selected, linguistically diverse subsets can match full multilingual alignment, and even outperform it for unseen LRLs. This indicates that effective realignment does not require exhaustive language coverage and can reduce data collection overhead, while remaining both efficient and robust when guided by informed language selection.