# Unsupervised Alignment of Embeddings with Wasserstein Procrustes

arXiv: https://arxiv.org/abs/1805.11222 · HTML: https://mlsanity.com/paper/1805.11222/

## Key facts

- Cited by 97 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 May 2018.

## Built on by (most cited first, 30 of 97)

- [Word Alignment by Fine-tuning Embeddings on Parallel Corpora](https://mlsanity.com/paper/2101.08231/) (2021)
- [Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions](https://mlsanity.com/paper/2209.03430/) (2022)
- [Unbalanced Optimal Transport, from Theory to Numerics](https://mlsanity.com/paper/2211.08775/) (2022)
- [Black Box Few-Shot Adaptation for Vision-Language models](https://mlsanity.com/paper/2304.01752/) (2023)
- [Pre-training for Speech Translation: CTC Meets Optimal Transport](https://mlsanity.com/paper/2301.11716/) (2023)
- [Random Graph Matching in Geometric Models: the Case of Complete Graphs](https://mlsanity.com/paper/2202.10662/) (2022)
- [Using Optimal Transport as Alignment Objective for fine-tuning Multilingual Contextualized Embeddings](https://mlsanity.com/paper/2110.02887/) (2021)
- [Discovering Universal Geometry in Embeddings with ICA](https://mlsanity.com/paper/2305.13175/) (2023)
- [An improved central limit theorem and fast convergence rates for entropic transportation costs](https://mlsanity.com/paper/2204.09105/) (2022)
- [Soft Matching Distance: A metric on neural representations that captures single-neuron tuning](https://mlsanity.com/paper/2311.09466/) (2023)
- [TADA: Task-Agnostic Dialect Adapters for English](https://mlsanity.com/paper/2305.16651/) (2023)
- [Entropic Gromov-Wasserstein between Gaussian Distributions](https://mlsanity.com/paper/2108.10961/) (2021)
- [Lower Complexity Adaptation for Empirical Entropic Optimal Transport](https://mlsanity.com/paper/2306.13580/) (2023)
- [Scalable Optimal Transport in High Dimensions for Graph Distances, Embedding Alignment, and More](https://mlsanity.com/paper/2107.06876/) (2021)
- [Re-evaluating Word Mover's Distance](https://mlsanity.com/paper/2105.14403/) (2021)
- [Towards an Explainable Comparison and Alignment of Feature Embeddings](https://mlsanity.com/paper/2506.06231/) (2025)
- [CreoleVal: Multilingual Multitask Benchmarks for Creoles](https://mlsanity.com/paper/2310.19567/) (2023)
- [Spectral Augmentations for Graph Contrastive Learning](https://mlsanity.com/paper/2302.02909/) (2023)
- [Structured Transforms Across Spaces with Cost-Regularized Optimal Transport](https://mlsanity.com/paper/2311.05788/) (2023)
- [Functional optimal transport: map estimation and domain adaptation for functional data](https://mlsanity.com/paper/2102.03895/) (2021)
- [Improving Bilingual Lexicon Induction with Cross-Encoder Reranking](https://mlsanity.com/paper/2210.16953/) (2022)
- [Bilingual Lexicon Induction for Low-Resource Languages using Graph Matching via Optimal Transport](https://mlsanity.com/paper/2210.14378/) (2022)
- [Mirror Sinkhorn: Fast Online Optimization on Transport Polytopes](https://mlsanity.com/paper/2211.10420/) (2022)
- [Quantized Wasserstein Procrustes Alignment of Word Embedding Spaces](https://mlsanity.com/paper/2212.02468/) (2022)
- [Outlier-Robust Gromov-Wasserstein for Graph Data](https://mlsanity.com/paper/2302.04610/) (2023)
- [Uncovering Challenges of Solving the Continuous Gromov-Wasserstein Problem](https://mlsanity.com/paper/2303.05978/) (2023)
- [Unbalanced Optimal Transport for Unbalanced Word Alignment](https://mlsanity.com/paper/2306.04116/) (2023)
- [Cycle Consistent Probability Divergences Across Different Spaces](https://mlsanity.com/paper/2111.11328/) (2021)
- [Unsupervised Manifold Alignment with Joint Multidimensional Scaling](https://mlsanity.com/paper/2207.02968/) (2022)
- [Personalised Federated Learning On Heterogeneous Feature Spaces](https://mlsanity.com/paper/2301.11447/) (2023)

## Abstract (arXiv)

We consider the task of aligning two sets of points in high dimension, which has many applications in natural language processing and computer vision. As an example, it was recently shown that it is possible to infer a bilingual lexicon, without supervised data, by aligning word embeddings trained on monolingual data. These recent advances are based on adversarial training to learn the mapping between the two embeddings. In this paper, we propose to use an alternative formulation, based on the joint estimation of an orthogonal matrix and a permutation matrix. While this problem is not convex, we propose to initialize our optimization algorithm by using a convex relaxation, traditionally considered for the graph isomorphism problem. We propose a stochastic algorithm to minimize our cost function on large scale problems. Finally, we evaluate our method on the problem of unsupervised word translation, by aligning word embeddings trained on monolingual data. On this task, our method obtains state of the art results, while requiring less computational resources than competing approaches.