mlsanityData to 13 Oct 2026

Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignment

arXiv 2101.00148 ↗First posted 1 Jan 2021cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

Bilingual lexicons map words in one language to their translations in another, and are typically induced by learning linear projections to align monolingual word embedding spaces. In this paper, we show it is possible to produce much higher quality lexicons with methods that combine (1) unsupervised bitext mining and (2) unsupervised word alignment. Directly applying a pipeline that uses recent algorithms for both subproblems significantly improves induced lexicon quality and further gains are possible by learning to filter the resulting lexical entries, with both unsupervised and semi-supervised schemes. Our final model outperforms the state of the art on the BUCC 2020 shared task by 14 F1 points averaged over 12 language pairs, while also providing a more interpretable approach that allows for rich reasoning of word meaning in context. Further analysis of our output and the standard reference lexicons suggests they are of comparable quality, and new benchmarks may be needed to measure further progress on this task.

Key facts. Cited by 19 papers in the map (data through 13 Oct 2026). Builds on 29 of its 49 references that are in the map. First posted on arXiv 1 Jan 2021.

Builds on

29 of 49 references.

References · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,093
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,916
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,893
  5. 2017Billion-scale similarity search with GPUscited by 2,490
  6. 2016Enriching Word Vectors with Subword Informationcited by 1,581
  7. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,033
  8. 2017Word Translation Without Parallel Datacited by 483
  9. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  10. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 221
  11. 2013Exploiting Similarities among Languages for Machine Translationcited by 190
  12. 2020SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddingscited by 155
17 more
  1. 2018A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddingscited by 146
  2. 2018Gromov-Wasserstein Alignment of Word Embedding Spacescited by 120
  3. 2017Offline bilingual word vectors, orthogonal transformations and the inverted softmaxcited by 95
  4. 2018Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddingscited by 83
  5. 2019Jointly Learning to Align and Translate with Transformer Modelscited by 75
  6. 2018Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterioncited by 70
  7. 2018On the Limitations of Unsupervised Bilingual Dictionary Inductioncited by 61
  8. 2020Cross-lingual Retrieval for Iterative Self-Supervised Trainingcited by 54
  9. 2014Improving zero-shot learning by mitigating the hubness problemcited by 53
  10. 2018Effective Parallel Corpus Mining using Bilingual Sentence Embeddingscited by 39
  11. 2018Filtering and Mining Parallel Data in a Joint Multilingual Spacecited by 32
  12. 2019Adding Interpretable Attention to Neural Translation Models Improves Word Alignmentcited by 30
  13. 2018Non-Adversarial Unsupervised Word Translationcited by 19
  14. 2019Bilingual Lexicon Induction through Unsupervised Machine Translationcited by 16
  15. 2020Unsupervised Bitext Mining and Translation via Self-trained Contextual Embeddingscited by 16
  16. 2019Lost in Evaluation: Misleading Benchmarks for Bilingual Dictionary Inductioncited by 10
  17. 2017An Empirical Analysis of NMT-Derived Interlingual Embeddings and their Use in Parallel Sentence Identificationcited by 9

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 5; 2023: 5; 2024: 5; 2025: 1.

19 papers.

Sort:

By area:

  1. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 525
  2. 2023Dictionary-based Phrase-level Prompting of Large Language Models for Machine Translationcited by 74
  3. 2022Improving Word Translation via Two-Stage Contrastive Learningcited by 18
  4. 2021Combining Static Word Embeddings and Contextual Representations for Bilingual Lexicon Inductioncited by 12
  5. 2023Multilingual Sentence Transformer as A Multilingual Word Alignercited by 8
  6. 2022Improving Bilingual Lexicon Induction with Cross-Encoder Rerankingcited by 7
  7. 2022Bilingual Lexicon Induction for Low-Resource Languages using Graph Matching via Optimal Transportcited by 6
  8. 2025The Mechanistic Emergence of Symbol Grounding in Language Modelscited by 6
  9. 2021Substructure Distribution Projection for Zero-Shot Cross-Lingual Dependency Parsingcited by 5
  10. 2021Graph Algorithms for Multiparallel Word Alignmentcited by 4
  11. 2024Tomato, Tomahto, Tomate: Do Multilingual Language Models Understand Based on Subword-Level Semantic Concepts?cited by 4
  12. 2023Does mBERT understand Romansh? Evaluating word embeddings using word alignmentcited by 3
7 more
  1. 2023Low-resource Bilingual Dialect Lexicon Induction with Large Language Modelscited by 2
  2. 2023ProMap: Effective Bilingual Lexicon Induction via Language Model Promptingcited by 2
  3. 2024Learning Language Structures through Groundingcited by 2
  4. 2022Can Synthetic Translations Improve Bitext Quality?cited by 1
  5. 2024How Lexical is Bilingual Lexicon Induction?cited by 0
  6. 2024A survey of neural-network-based methods utilising comparable data for finding translation equivalentscited by 0
  7. 2024Unsupervised Bilingual Lexicon Induction for Low Resource Languagescited by 0