mlsanityData to 7 Oct 2026

Combining Static Word Embeddings and Contextual Representations for Bilingual Lexicon Induction

arXiv 2106.03084 ↗First posted 6 Jun 2021cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

Bilingual Lexicon Induction (BLI) aims to map words in one language to their translations in another, and is typically through learning linear projections to align monolingual word representation spaces. Two classes of word representations have been explored for BLI: static word embeddings and contextual representations, but there is no studies to combine both. In this paper, we propose a simple yet effective mechanism to combine the static word embeddings and the contextual representations to utilize the advantages of both paradigms. We test the combination mechanism on various language pairs under the supervised and unsupervised BLI benchmark settings. Experiments show that our mechanism consistently improves performances over robust BLI baselines on all language pairs by averagely improving 3.2 points in the supervised setting, and 3.1 points in the unsupervised setting.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 23 of its 43 references that are in the map. First posted on arXiv 6 Jun 2021.

Builds on

23 of 43 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  3. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  4. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  5. 2017Word Translation Without Parallel Datacited by 480
  6. 2013Exploiting Similarities among Languages for Machine Translationcited by 187
  7. 2018A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddingscited by 145
  8. 2017A Survey Of Cross-lingual Word Embedding Modelscited by 142
  9. 2018When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?cited by 125
  10. 2020Multilingual Alignment of Contextual Word Representationscited by 113
  11. 2017Offline bilingual word vectors, orthogonal transformations and the inverted softmaxcited by 94
  12. 2018Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterioncited by 69
11 more
  1. 2019Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsingcited by 69
  2. 2018On the Limitations of Unsupervised Bilingual Dictionary Inductioncited by 61
  3. 2014Improving zero-shot learning by mitigating the hubness problemcited by 53
  4. 2019Bilingual Lexicon Induction with Semi-supervision in Non-Isometric Embedding Spacescited by 35
  5. 2019Cross-lingual Alignment vs Joint Training: A Comparative Study and A Simple Unified Frameworkcited by 26
  6. 2020Cross-lingual Alignment Methods for Multilingual BERT: A Comparative Studycited by 24
  7. 2020LNMap: Departures from Isomorphic Assumption in Bilingual Lexicon Induction Through Non-Linear Mapping in Latent Spacecited by 23
  8. 2021Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignmentcited by 19
  9. 2019Context-Aware Cross-Lingual Mappingcited by 17
  10. 2019Bilingual Lexicon Induction through Unsupervised Machine Translationcited by 16
  11. 2019Revisiting Adversarial Autoencoder for Unsupervised Word Translation with Cycle Consistency and Improved Trainingcited by 12

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 4; 2024: 3; 2025: 2.

12 papers.

  1. 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Biascited by 82
  2. 2023Hyperpolyglot LLMs: Cross-Lingual Interpretability in Token Embeddingscited by 19
  3. 2022Improving Word Translation via Two-Stage Contrastive Learningcited by 18
  4. 2022Improving Bilingual Lexicon Induction with Cross-Encoder Rerankingcited by 7
  5. 2023When your Cousin has the Right Connections: Unsupervised Bilingual Lexicon Induction for Related Data-Imbalanced Languagescited by 3
  6. 2023On Bilingual Lexicon Induction with Large Language Modelscited by 2
  7. 2023ProMap: Effective Bilingual Lexicon Induction via Language Model Promptingcited by 2
  8. 2022Leveraging Emotion-specific Features to Improve Transformer Performance for Emotion Classificationcited by 0
  9. 2024A survey of neural-network-based methods utilising comparable data for finding translation equivalentscited by 0
  10. 2024Unsupervised Bilingual Lexicon Induction for Low Resource Languagescited by 0
  11. 2025Cross-Domain Bilingual Lexicon Induction via Pretrained Language Modelscited by 0
  12. 2025How Good is BLI as an Alignment Measure: A Study in Word Embedding Paradigmcited by 0