# Franken-Adapter: Cross-Lingual Adaptation of LLMs by Embedding Surgery

arXiv: https://arxiv.org/abs/2502.08037 · HTML: https://mlsanity.com/paper/2502.08037/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Feb 2025.

## Built on by (most cited first, 5 of 5)

- [The Unreasonable Effectiveness of Model Merging for Cross-Lingual Transfer in LLMs](https://mlsanity.com/paper/2505.18356/) (2025)
- [Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling](https://mlsanity.com/paper/2604.25578/) (2026)
- [Multilingual Unlearning in LLMs: Transfer, Dynamics, and Reversibility](https://mlsanity.com/paper/2606.03291/) (2026)
- [Do LLMs Need Inherent Reasoning Before Reinforcement Learning? A Study in Korean Self-Correction](https://mlsanity.com/paper/2601.05459/) (2026)
- [COMPASS: COntinual Multilingual PEFT with Adaptive Semantic Sampling](https://mlsanity.com/paper/2604.20720/) (2026)

## Abstract (arXiv)

The capabilities of Large Language Models (LLMs) in low-resource languages lag far behind those in English, making their universal accessibility a significant challenge. To alleviate this, we present textitFranken-Adapter, a modular language adaptation approach for decoder-only LLMs with embedding surgery. Our method begins by creating customized vocabularies for target languages and performing language adaptation through embedding tuning on multilingual data. These pre-trained embeddings are subsequently integrated with LLMs that have been instruction-tuned on English alignment data to enable zero-shot cross-lingual transfer. Our experiments on textttGemma2 models with up to 27B parameters demonstrate improvements of up to 20% across 96 languages, spanning both discriminative and generative tasks, with minimal regressions (<1%) in English. Further in-depth analysis reveals the critical role of customizing tokenizers in enhancing language adaptation, while boosting inference efficiency. Additionally, we show the versatility of our method by achieving a 14% improvement over a math-optimized LLM across 20 languages, offering a modular solution to transfer reasoning abilities across languages post hoc.