mlsanityData to 7 Oct 2026

What does it mean to be language-agnostic? Probing multilingual sentence encoders for typological properties

arXiv 2009.12862 ↗First posted 27 Sep 2020cs.CLcited by 23

Not on the map. Why?

Abstract (arXiv)

Multilingual sentence encoders have seen much success in cross-lingual model transfer for downstream NLP tasks. Yet, we know relatively little about the properties of individual languages or the general patterns of linguistic variation that they encode. We propose methods for probing sentence representations from state-of-the-art multilingual encoders (LASER, M-BERT, XLM and XLM-R) with respect to a range of typological properties pertaining to lexical, morphological and syntactic structure. In addition, we investigate how this information is distributed across all layers of the models. Our results show interesting differences in encoding linguistic variation associated with different pretraining strategies.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 6; 2023: 4; 2024: 4; 2025: 0; 2026: 1.

23 papers.

Sort:

By area:

  1. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 275
  2. 2022The Geometry of Multilingual Language Model Representationscited by 94
  3. 2021A Primer on Pretrained Multilingual Language Modelscited by 49
  4. 2024Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformerscited by 42
  5. 2022Metaphors in Pre-Trained Language Models: Probing and Generalization Across Datasets and Languagescited by 30
  6. 2022Discovering Language-neutral Sub-networks in Multilingual Language Modelscited by 28
  7. 2021Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids' Representationscited by 16
  8. 2022Cross-Linguistic Syntactic Difference in Multilingual BERT: How Good is It and How Does It Affect Transfer?cited by 10
  9. 2021Does Typological Blinding Impede Cross-Lingual Sharing?cited by 9
  10. 2021On the Language-specificity of Multilingual BERT and the Impact of Fine-tuningcited by 8
  11. 2022Are Pretrained Multilingual Models Equally Fair Across Languages?cited by 8
  12. 2021Locating Language-Specific Information in Contextualized Embeddingscited by 6
11 more
  1. 2023A Joint Matrix Factorization Analysis of Multilingual Representationscited by 5
  2. 2024Analysis of Multi-Source Language Training in Cross-Lingual Transfercited by 5
  3. 2021RuSentEval: Linguistic Source, Encoder Force!cited by 4
  4. 2021Discovering Representation Sprachbund For Multilingual Pre-Trainingcited by 4
  5. 2022Multilingual Transformer Encoders: a Word-Level Task-Agnostic Evaluationcited by 4
  6. 2023The Less the Merrier? Investigating Language Representation in Multilingual Modelscited by 2
  7. 2023The Grammar and Syntax Based Corpus Analysis Tool For The Ukrainian Languagecited by 1
  8. 2023Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language Modelscited by 1
  9. 2021Do Multi-Lingual Pre-trained Language Models Reveal Consistent Token Attributions in Different Languages?cited by 0
  10. 2024The correlation between nativelike selection and prototypicality: a multilingual onomasiological case study using semantic embeddingcited by 0
  11. 2026When Language Representations Interact: Separability and Cross-Lingual Effects in LLMscited by 0