mlsanityData to 7 Oct 2026

IsoVec: Controlling the Relative Isomorphism of Word Embedding Spaces

arXiv 2210.05098 ↗First posted 11 Oct 2022cs.CL cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

The ability to extract high-quality translation dictionaries from monolingual word embedding spaces depends critically on the geometric similarity of the spaces -- their degree of "isomorphism." We address the root-cause of faulty cross-lingual mapping: that word embedding training resulted in the underlying spaces being non-isomorphic. We incorporate global measures of isomorphism directly into the Skip-gram loss function, successfully increasing the relative isomorphism of trained word embedding spaces and improving their ability to be mapped to a shared cross-lingual space. The result is improved bilingual lexicon induction in general data conditions, under domain mismatch, and with training algorithm dissimilarities. We release IsoVec at https://github.com/kellymarchisio/isovec.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 42 of its 68 references that are in the map. First posted on arXiv 11 Oct 2022.

Builds on

42 of 68 references.

References · 42

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  5. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  6. 2017Word Translation Without Parallel Datacited by 480
  7. 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
  8. 2017All-but-the-Top: Simple and Effective Postprocessing for Word Representationscited by 205
  9. 2021Whitening Sentence Representations for Better Semantics and Faster Retrievalcited by 198
  10. 2018A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddingscited by 145
  11. 2017A Survey Of Cross-lingual Word Embedding Modelscited by 142
  12. 2018Gromov-Wasserstein Alignment of Word Embedding Spacescited by 119
30 more
  1. 2020Multilingual Alignment of Contextual Word Representationscited by 113
  2. 2015Optimal whitening and decorrelationcited by 93
  3. 2018Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterioncited by 69
  4. 2018On the Limitations of Unsupervised Bilingual Dictionary Inductioncited by 61
  5. 2021IsoScore: Measuring the Uniformity of Embedding Space Utilizationcited by 52
  6. 2019How to (Properly) Evaluate Cross-Lingual Word Embeddings: On Strong Baselines, Comparative Analyses, and Some Misconceptionscited by 48
  7. 2018FRAGE: Frequency-Agnostic Word Representationcited by 47
  8. 2019Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsingcited by 46
  9. 2020When Does Unsupervised Machine Translation Work?cited by 38
  10. 2020Are All Good Word Vector Spaces Isomorphic?cited by 36
  11. 2019Bilingual Lexicon Induction with Semi-supervision in Non-Isometric Embedding Spacescited by 35
  12. 2020When and Why is Unsupervised Neural Machine Translation Useless?cited by 34
  13. 2019Do We Really Need Fully Unsupervised Cross-Lingual Embeddings?cited by 31
  14. 2021Attention Flows are Shapley Value Explanationscited by 30
  15. 2019Cross-lingual Alignment vs Joint Training: A Comparative Study and A Simple Unified Frameworkcited by 26
  16. 2021An Isotropy Analysis in the Multilingual BERT Embedding Spacecited by 25
  17. 2020Cross-lingual Alignment Methods for Multilingual BERT: A Comparative Studycited by 24
  18. 2020LNMap: Departures from Isomorphic Assumption in Bilingual Lexicon Induction Through Non-Linear Mapping in Latent Spacecited by 23
  19. 2018Learning Multilingual Word Embeddings in Latent Metric Space: A Geometric Approachcited by 22
  20. 2019Analyzing the Limitations of Cross-lingual Word Embedding Mappingscited by 18
  21. 2022Improving Word Translation via Two-Stage Contrastive Learningcited by 18
  22. 2014BilBOWA: Fast Bilingual Distributed Representations without Word Alignmentscited by 16
  23. 2019Are Girls Neko or Sh=ojo? Cross-Lingual Alignment of Non-Isomorphic Embeddings with Iterative Normalizationcited by 15
  24. 2019Don't Forget the Long Tail! A Comprehensive Analysis of Morphological Generalization in Bilingual Lexicon Inductioncited by 15
  25. 2016Towards cross-lingual distributed representations without parallel text trained with adversarial autoencoderscited by 13
  26. 2018Improving Cross-Lingual Word Embeddings by Meeting in the Middlecited by 13
  27. 2020Beyond Offline Mapping: Learning Cross Lingual Word Embeddings through Context Anchoringcited by 8
  28. 2018Characterizing Departures from Linearity in Word Translationcited by 7
  29. 2020Anchor-based Bilingual Word Embeddings for Low-Resource Languagescited by 6
  30. 2021Cross-Lingual Word Embedding Refinement by ℓ1 Norm Optimisationcited by 6

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 1; 2025: 2.

7 papers.

  1. 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Biascited by 82
  2. 2023GRI: Graph-based Relative Isomorphism of Word Embedding Spacescited by 3
  3. 2023GARI: Graph Attention for Relative Isomorphism of Arabic Word Embeddingscited by 3
  4. 2023On Bilingual Lexicon Induction with Large Language Modelscited by 2
  5. 2025Words as Bridges: Exploring Computational Support for Cross-Disciplinary Translation Workcited by 2
  6. 2025Hierarchical Level-Wise News Article Clustering via Multilingual Matryoshka Embeddingscited by 2
  7. 2023Accessing Higher Dimensions for Unsupervised Word Translationcited by 0