mlsanityData to 14 Oct 2026

A Survey Of Cross-lingual Word Embedding Models

arXiv 1706.04902 ↗First posted 15 Jun 2017cs.CL cs.LGcited by 143

Not on the map. Why?

Abstract (arXiv)

Cross-lingual representations of words enable us to reason about word meaning in multilingual contexts and are a key facilitator of cross-lingual transfer when developing natural language processing models for low-resource languages. In this survey, we provide a comprehensive typology of cross-lingual word embedding models. We compare their data requirements and objective functions. The recurring theme of the survey is that many of the models presented in the literature optimize for the same objectives, and that seemingly different models are often equivalent modulo optimization strategies, hyper-parameters, and such. We also discuss the different ways cross-lingual word embeddings are evaluated, as well as future challenges and research horizons.

Key facts. Cited by 143 papers in the map (data through 14 Oct 2026). First posted on arXiv 15 Jun 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 51; 2022: 28; 2023: 27; 2024: 16; 2025: 13; 2026: 8.

143 papers.

Sort:

By area:

  1. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  2. 2023ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learningcited by 206
  3. 2023Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Modelscited by 148
  4. 2021It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoningcited by 69
  5. 2021First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERTcited by 65
  6. 2022A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challengescited by 61
  7. 2021How to Adapt Your Pretrained Multilingual Model to 1600 Languagescited by 54
  8. 2021A Survey on Green Deep Learningcited by 52
  9. 2021A Primer on Pretrained Multilingual Language Modelscited by 49
  10. 2021Revisiting the Primacy of English in Zero-shot Cross-lingual Transfercited by 47
  11. 2022Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptationcited by 44
  12. 2022Models and Datasets for Cross-Lingual Summarisationcited by 36
38 more
  1. 2022Enhancing Cross-lingual Transfer by Manifold Mixupcited by 30
  2. 2021On Cross-Lingual Retrieval with Multilingual Text Encoderscited by 22
  3. 2021Using Optimal Transport as Alignment Objective for fine-tuning Multilingual Contextualized Embeddingscited by 19
  4. 2023Hyperpolyglot LLMs: Cross-Lingual Interpretability in Token Embeddingscited by 19
  5. 2024Recent advances in text embedding: A Comprehensive Review of Top-Performing Methods on the MTEB Benchmarkcited by 19
  6. 2021COVID-19 sentiment analysis via deep learning during the rise of novel casescited by 18
  7. 2021XLM-K: Improving Cross-Lingual Language Model Pre-training with Multilingual Knowledgecited by 18
  8. 2022Improving Word Translation via Two-Stage Contrastive Learningcited by 18
  9. 2021Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systemscited by 17
  10. 2021Should we Stop Training More Monolingual Models, and Simply Use Machine Translation Instead?cited by 17
  11. 2022Cross-Lingual Ability of Multilingual Masked Language Models: A Study of Language Structurecited by 17
  12. 2024Abstractive Text Summarization: State of the Art, Challenges, and Improvementscited by 17
  13. 2021Evaluating Multilingual Text Encoders for Unsupervised Cross-Lingual Retrievalcited by 16
  14. 2021Modelling Latent Translations for Cross-Lingual Transfercited by 16
  15. 2022Learning Backward Compatible Embeddingscited by 16
  16. 2023SLABERT Talk Pretty One Day: Modeling Second Language Acquisition with BERTcited by 15
  17. 2021Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translationcited by 14
  18. 2021Cross-lingual Transfer of Monolingual Modelscited by 14
  19. 2024Tokenization and Morphology in Multilingual Language Models: A Comparative Analysis of mT5 and ByT5cited by 14
  20. 2021MergeDistill: Merging Pre-trained Language Models using Distillationcited by 13
  21. 2021conSultantBERT: Fine-tuned Siamese Sentence-BERT for Matching Jobs and Job Seekerscited by 13
  22. 2021Combining Static Word Embeddings and Contextual Representations for Bilingual Lexicon Inductioncited by 12
  23. 2025Tokenization Matters: Improving Zero-Shot NER for Indic Languagescited by 12
  24. 2021Scalable Optimal Transport in High Dimensions for Graph Distances, Embedding Alignment, and Morecited by 11
  25. 2023Language Models are Universal Embedderscited by 11
  26. 2024Why do LLaVA Vision-Language Models Reply to Images in English?cited by 11
  27. 2021Code-switched inspired losses for generic spoken dialog representationscited by 10
  28. 2023Learning to Speak from Text: Zero-Shot Multilingual Text-to-Speech with Unsupervised Text Pretrainingcited by 10
  29. 2023Massively Multilingual Corpus of Sentiment Datasets and Multi-faceted Sentiment Classification Benchmarkcited by 10
  30. 2021AllWOZ: Towards Multilingual Task-Oriented Dialog Systems for Allcited by 9
  31. 2023Beyond Shared Vocabulary: Increasing Representational Word Similarities across Languages for Multilingual Machine Translationcited by 9
  32. 2021Bootstrapping Multilingual AMR with Contextual Word Alignmentscited by 8
  33. 2021Fine-tuning Encoders for Improved Monolingual and Zero-shot Polylingual Neural Topic Modelingcited by 8
  34. 2021On the Language-specificity of Multilingual BERT and the Impact of Fine-tuningcited by 8
  35. 2022Por Qu'e N~ao Utiliser Alla Spraak? Mixed Training with Gradient Optimization in Few-Shot Cross-Lingual Transfercited by 7
  36. 2022IsoVec: Controlling the Relative Isomorphism of Word Embedding Spacescited by 7
  37. 2022Improving Bilingual Lexicon Induction with Cross-Encoder Rerankingcited by 7
  38. 2023μPLAN: Summarizing using a Content Plan as Cross-Lingual Bridgecited by 7

50 most cited of 143 All 143 (JSON)