# What makes multilingual BERT multilingual?

arXiv: https://arxiv.org/abs/2010.10938 · HTML: https://mlsanity.com/paper/2010.10938/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2020.

## Built on by (most cited first, 1 of 1)

- [Assessment of Massively Multilingual Sentiment Classifiers](https://mlsanity.com/paper/2204.04937/) (2022)

## Abstract (arXiv)

Recently, multilingual BERT works remarkably well on cross-lingual transfer tasks, superior to static non-contextualized word embeddings. In this work, we provide an in-depth experimental study to supplement the existing literature of cross-lingual ability. We compare the cross-lingual ability of non-contextualized and contextualized representation model with the same data. We found that datasize and context window size are crucial factors to the transferability.