# Are Girls Neko or Sh=ojo? Cross-Lingual Alignment of Non-Isomorphic Embeddings with Iterative Normalization

arXiv: https://arxiv.org/abs/1906.01622 · HTML: https://mlsanity.com/paper/1906.01622/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jun 2019.

## Built on by (most cited first, 15 of 15)

- [Optimization of Graph Neural Networks: Implicit Acceleration by Skip Connections and More Depth](https://mlsanity.com/paper/2105.04550/) (2021)
- [CiCo: Domain-Aware Sign Language Retrieval via Cross-Lingual Contrastive Learning](https://mlsanity.com/paper/2303.12793/) (2023)
- [Learning Backward Compatible Embeddings](https://mlsanity.com/paper/2206.03040/) (2022)
- [A Massively Multilingual Analysis of Cross-linguality in Shared Embedding Space](https://mlsanity.com/paper/2109.06324/) (2021)
- [IsoVec: Controlling the Relative Isomorphism of Word Embedding Spaces](https://mlsanity.com/paper/2210.05098/) (2022)
- [Cross-Lingual BERT Contextual Embedding Space Mapping with Isotropic and Isometric Conditions](https://mlsanity.com/paper/2107.09186/) (2021)
- [Bilingual Lexicon Induction for Low-Resource Languages using Graph Matching via Optimal Transport](https://mlsanity.com/paper/2210.14378/) (2022)
- [Zero-Shot Cross-Lingual Dependency Parsing through Contextual Embedding Transformation](https://mlsanity.com/paper/2103.02212/) (2021)
- [Majority Voting with Bidirectional Pre-translation For Bitext Retrieval](https://mlsanity.com/paper/2103.06369/) (2021)
- [RAPO: An Adaptive Ranking Paradigm for Bilingual Lexicon Induction](https://mlsanity.com/paper/2210.09926/) (2022)
- [GRI: Graph-based Relative Isomorphism of Word Embedding Spaces](https://mlsanity.com/paper/2310.12360/) (2023)
- [GARI: Graph Attention for Relative Isomorphism of Arabic Word Embeddings](https://mlsanity.com/paper/2310.13068/) (2023)
- [An Analysis of Euclidean vs. Graph-Based Framing for Bilingual Lexicon Induction from Word Embedding Spaces](https://mlsanity.com/paper/2109.12640/) (2021)
- [Hierarchical Level-Wise News Article Clustering via Multilingual Matryoshka Embeddings](https://mlsanity.com/paper/2506.00277/) (2025)
- [Towards Efficient Methods in Medical Question Answering using Knowledge Graph Embeddings](https://mlsanity.com/paper/2401.07977/) (2024)

## Abstract (arXiv)

Cross-lingual word embeddings (CLWE) underlie many multilingual natural language processing systems, often through orthogonal transformations of pre-trained monolingual embeddings. However, orthogonal mapping only works on language pairs whose embeddings are naturally isomorphic. For non-isomorphic pairs, our method (Iterative Normalization) transforms monolingual embeddings to make orthogonal alignment easier by simultaneously enforcing that (1) individual word vectors are unit length, and (2) each language's average vector is zero. Iterative Normalization consistently improves word translation accuracy of three CLWE methods, with the largest improvement observed on English-Japanese (from 2% to 44% test accuracy).