mlsanityData to 7 Oct 2026

Cross-lingual Representation Learning via Centroid Intervention Fusion

arXiv 2608.26357 ↗First posted 26 Aug 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) exhibit uneven multilingual performance, especially when dealing with low-resource languages. Inference-time intervention offers a lightweight way to improve cross-lingual transfer by modifying the hidden states produced by the LLMs during the forward pass, without updating model parameters. However, existing cross-lingual intervention methods typically learn separate projections from source to target languages, which limits scalability and prevents knowledge sharing across languages. We propose Centroid Intervention Fusion (CIF), a projection fusion framework that consolidates multiple multilingual intervention projections into a single language-shared operator. Across multilingual commonsense reasoning, natural language inference, factual editing, and machine translation benchmarks, CIF outperforms the strongest prior pairwise intervention baseline by up to +3.378 pp on average across four model backbones, while supporting performance gains for low resource languages. The code is available at https://github.com/VRCMF/CIF.git.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 30 of its 41 references that are in the map. First posted on arXiv 26 Aug 2026.

Builds on

30 of 41 references.

References · 30

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  4. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  5. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  6. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  7. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  8. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  9. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  10. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  11. 2023Steering Language Models With Activation Engineeringcited by 824
  12. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 698
18 more
  1. 2019How multilingual is Multilingual BERT?cited by 642
  2. 2020Language-agnostic BERT Sentence Embeddingcited by 621
  3. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  4. 2017Word Translation Without Parallel Datacited by 480
  5. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  6. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 330
  7. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  8. 2020Are All Languages Created Equal in Multilingual BERT?cited by 206
  9. 2013Exploiting Similarities among Languages for Machine Translationcited by 187
  10. 2018A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddingscited by 145
  11. 2019CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairscited by 136
  12. 2020On the Language Neutrality of Pre-trained Multilingual Representationscited by 82
  13. 2019Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsingcited by 69
  14. 2021It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoningcited by 69
  15. 2021Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoningcited by 64
  16. 2024Semantics-Adaptive Activation Intervention for LLMs via Dynamic Steering Vectorscited by 61
  17. 2020Inducing Language-Agnostic Multilingual Representationscited by 48
  18. 2024Bridging the Language Gaps in Large Language Models with Inference-Time Cross-Lingual Interventioncited by 24

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Gradual Code-Switching as Inference-Time Cross-Lingual Representational Alignment for LLMs
  2. 2025Cross-Lingual Consistency: A Novel Inference Framework for Advancing Reasoning in Large Language Models
  3. 2025Rethinking Cross-lingual Alignment: Balancing Transfer and Cultural Erasure in Multilingual LLMs
  4. 2024CrossIn: An Efficient Instruction Tuning Approach for Cross-Lingual Knowledge Alignment
  5. 2025The Unreasonable Effectiveness of Model Merging for Cross-Lingual Transfer in LLMs
  6. 2025Unlocking Multilingual Reasoning Capability of LLMs and LVLMs through Representation Engineering
  7. 2024Lens: Rethinking Multilingual Enhancement for Large Language Models
  8. 2025Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMs