# Embedding-Enhanced Giza++: Improving Alignment in Low- and High- Resource Scenarios Using Embedding Space Geometry

arXiv: https://arxiv.org/abs/2104.08721 · HTML: https://mlsanity.com/paper/2104.08721/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 42 references that are in the map.
- First posted on arXiv 18 Apr 2021.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [Word Alignment by Fine-tuning Embeddings on Parallel Corpora](https://mlsanity.com/paper/2101.08231/) (2021)
- [SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings](https://mlsanity.com/paper/2004.08728/) (2020)
- [A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings](https://mlsanity.com/paper/1805.06297/) (2018)
- [Jointly Learning to Align and Translate with Transformer Models](https://mlsanity.com/paper/1909.02074/) (2019)
- [An Effective Approach to Unsupervised Machine Translation](https://mlsanity.com/paper/1902.01313/) (2019)
- [Bilingual Lexicon Induction with Semi-supervision in Non-Isometric Embedding Spaces](https://mlsanity.com/paper/1908.06625/) (2019)
- [Accurate Word Alignment Induction from Neural Machine Translation](https://mlsanity.com/paper/2004.14837/) (2020)
- [Adding Interpretable Attention to Neural Translation Models Improves Word Alignment](https://mlsanity.com/paper/1901.11359/) (2019)
- [End-to-End Neural Word Alignment Outperforms GIZA++](https://mlsanity.com/paper/2004.14675/) (2020)
- [A Supervised Word Alignment Method based on Cross-Language Span Prediction using Multilingual BERT](https://mlsanity.com/paper/2004.14516/) (2020)
- [A Discriminative Neural Model for Cross-Lingual Word Alignment](https://mlsanity.com/paper/1909.00444/) (2019)
- [On a Novel Application of Wasserstein-Procrustes for Unsupervised Cross-Lingual Learning](https://mlsanity.com/paper/2007.09456/) (2020)
- [Learning to Relate from Captions and Bounding Boxes](https://mlsanity.com/paper/1912.00311/) (2019)

## Abstract (arXiv)

A popular natural language processing task decades ago, word alignment has been dominated until recently by GIZA++, a statistical method based on the 30-year-old IBM models. New methods that outperform GIZA++ primarily rely on large machine translation models, massively multilingual language models, or supervision from GIZA++ alignments itself. We introduce Embedding-Enhanced GIZA++, and outperform GIZA++ without any of the aforementioned factors. Taking advantage of monolingual embedding spaces of source and target language only, we exceed GIZA++'s performance in every tested scenario for three languages pairs. In the lowest-resource setting, we outperform GIZA++ by 8.5, 10.9, and 12 AER for Ro-En, De-En, and En-Fr, respectively. We release our code at https://github.com/kellymarchisio/ee-giza.