# End-to-End Neural Word Alignment Outperforms GIZA++

arXiv: https://arxiv.org/abs/2004.14675 · HTML: https://mlsanity.com/paper/2004.14675/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Apr 2020.

## Built on by (most cited first, 28 of 28)

- [Word Alignment by Fine-tuning Embeddings on Parallel Corpora](https://mlsanity.com/paper/2101.08231/) (2021)
- [A Primer on Pretrained Multilingual Language Models](https://mlsanity.com/paper/2107.00676/) (2021)
- [Frustratingly Easy Label Projection for Cross-lingual Transfer](https://mlsanity.com/paper/2211.15613/) (2022)
- [Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMT](https://mlsanity.com/paper/2109.01396/) (2021)
- [Translate & Fill: Improving Zero-Shot Multilingual Semantic Parsing with Synthetic Data](https://mlsanity.com/paper/2109.04319/) (2021)
- [Salute the Classic: Revisiting Challenges of Machine Translation in the Age of Large Language Models](https://mlsanity.com/paper/2401.08350/) (2024)
- [Neural semi-Markov CRF for Monolingual Word Alignment](https://mlsanity.com/paper/2106.02569/) (2021)
- [Inducing and Using Alignments for Transition-based AMR Parsing](https://mlsanity.com/paper/2205.01464/) (2022)
- [A Crosslingual Investigation of Conceptualization in 1335 Languages](https://mlsanity.com/paper/2305.08475/) (2023)
- [Integrating Vectorized Lexical Constraints for Neural Machine Translation](https://mlsanity.com/paper/2203.12210/) (2022)
- [Multilingual Sentence Transformer as A Multilingual Word Aligner](https://mlsanity.com/paper/2301.12140/) (2023)
- [To Translate or Not to Translate: A Systematic Investigation of Translation-Based Cross-Lingual Transfer to Low-Resource Languages](https://mlsanity.com/paper/2311.09404/) (2023)
- [Unbalanced Optimal Transport for Unbalanced Word Alignment](https://mlsanity.com/paper/2306.04116/) (2023)
- [MirrorAlign: A Super Lightweight Unsupervised Word Alignment Model via Cross-Lingual Contrastive Learning](https://mlsanity.com/paper/2102.04009/) (2021)
- [Graph Algorithms for Multiparallel Word Alignment](https://mlsanity.com/paper/2109.06283/) (2021)
- [Monotonic Simultaneous Translation with Chunk-wise Reordering and Refinement](https://mlsanity.com/paper/2110.09646/) (2021)
- [Meeting the Needs of Low-Resource Languages: The Value of Automatic Alignments via Pretrained Models](https://mlsanity.com/paper/2302.07912/) (2023)
- [ParCourE: A Parallel Corpus Explorer for a Massively Multilingual Corpus](https://mlsanity.com/paper/2107.06632/) (2021)
- [Checks and Strategies for Enabling Code-Switched Machine Translation](https://mlsanity.com/paper/2210.05096/) (2022)
- [Word Alignment in the Era of Deep Learning: A Tutorial](https://mlsanity.com/paper/2212.00138/) (2022)
- [BinaryAlign: Word Alignment as Binary Sequence Labeling](https://mlsanity.com/paper/2407.12881/) (2024)
- [Cross-Align: Modeling Deep Cross-lingual Interactions for Word Alignment](https://mlsanity.com/paper/2210.04141/) (2022)
- [Accurate Online Posterior Alignments for Principled Lexically-Constrained Decoding](https://mlsanity.com/paper/2204.00871/) (2022)
- [Noisy Parallel Data Alignment](https://mlsanity.com/paper/2301.09685/) (2023)
- [OTTAWA: Optimal TransporT Adaptive Word Aligner for Hallucination and Omission Translation Errors Detection](https://mlsanity.com/paper/2406.01919/) (2024)
- [Embedding-Enhanced Giza++: Improving Alignment in Low- and High- Resource Scenarios Using Embedding Space Geometry](https://mlsanity.com/paper/2104.08721/) (2021)
- [A Template-based Method for Constrained Neural Machine Translation](https://mlsanity.com/paper/2205.11255/) (2022)
- [Third-Party Aligner for Neural Word Alignments](https://mlsanity.com/paper/2211.04198/) (2022)

## Abstract (arXiv)

Word alignment was once a core unsupervised learning task in natural language processing because of its essential role in training statistical machine translation (MT) models. Although unnecessary for training neural MT models, word alignment still plays an important role in interactive applications of neural machine translation, such as annotation transfer and lexicon injection. While statistical MT methods have been replaced by neural approaches with superior performance, the twenty-year-old GIZA++ toolkit remains a key component of state-of-the-art word alignment systems. Prior work on neural word alignment has only been able to outperform GIZA++ by using its output during training. We present the first end-to-end neural word alignment method that consistently outperforms GIZA++ on three data sets. Our approach repurposes a Transformer model trained for supervised translation to also serve as an unsupervised word alignment model in a manner that is tightly integrated and does not affect translation quality.