mlsanityData to 7 Oct 2026

AlignFreeze: Navigating the Impact of Realignment on the Layers of Multilingual Models Across Diverse Languages

arXiv 2502.12959 ↗First posted 18 Feb 2025cs.CL cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Realignment techniques are often employed to enhance cross-lingual transfer in multilingual language models, still, they can sometimes degrade performance in languages that differ significantly from the fine-tuned source language. This paper introduces AlignFreeze, a method that freezes either the layers' lower half or upper half during realignment. Through controlled experiments on 4 tasks, 3 models, and in 35 languages, we find that realignment affects all the layers but can be the most detrimental to the lower ones. Freezing the lower layers can prevent performance degradation. Particularly, AlignFreeze improves Part-of-Speech (PoS) tagging performances in languages where full realignment fails: with XLM-R, it provides improvements of more than one standard deviation in accuracy in seven more languages than full realignment.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 23 of its 32 references that are in the map. First posted on arXiv 18 Feb 2025.

Builds on

23 of 32 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2018Deep contextualized word representationscited by 2,285
  5. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  6. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  7. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
  8. 2017Word Translation Without Parallel Datacited by 480
  9. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
  10. 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
  11. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  12. 2022CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Taskcited by 205
11 more
  1. 2021Word Alignment by Fine-tuning Embeddings on Parallel Corporacited by 186
  2. 2020SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddingscited by 155
  3. 2018A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddingscited by 145
  4. 2020Multilingual Alignment of Contextual Word Representationscited by 113
  5. 2020Inducing Language-Agnostic Multilingual Representationscited by 48
  6. 2019Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsingcited by 46
  7. 2020Multilingual BERT Post-Pretraining Alignmentcited by 30
  8. 2020A Supervised Word Alignment Method based on Cross-Language Span Prediction using Multilingual BERTcited by 27
  9. 2023Exploring the Relationship between Alignment and Cross-lingual Transfer in Multilingual Transformerscited by 18
  10. 2022Multilingual Transformer Encoders: a Word-Level Task-Agnostic Evaluationcited by 4
  11. 2023Meeting the Needs of Low-Resource Languages: The Value of Automatic Alignments via Pretrained Modelscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Harnessing Linguistic Dissimilarity for Language Generalization on Unseen Low-Resource Varietiescited by 0
  2. 2026Online Language Adaptive Sampling for Better Distributed Cross-lingual Gainscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Rethinking what Matters: Effective and Robust Multilingual Realignment for Low-Resource Languages
  2. 2024PreAlign: Boosting Cross-Lingual Transfer by Early Establishment of Multilingual Alignment
  3. 2025Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMs
  4. 2025The Realignment Problem: When Right becomes Wrong in LLMs
  5. 2026Cross-Lingual Alignment Without Joint Training: Do Monolingual Language Models Converge on Universal Representations?
  6. 2026Wrong Before Right: Late Rescue and Interface Failure in Aligned Language Models
  7. 2024Understanding Cross-Lingual Alignment -- A Survey
  8. 2024Decoding-time Realignment of Language Models