Word Alignment in the Era of Deep Learning: A Tutorial
Not on the map. Why?
Abstract (arXiv)
The word alignment task, despite its prominence in the era of statistical machine translation (SMT), is niche and under-explored today. In this two-part tutorial, we argue for the continued relevance for word alignment. The first part provides a historical background to word alignment as a core component of the traditional SMT pipeline. We zero-in on GIZA++, an unsupervised, statistical word aligner with surprising longevity. Jumping forward to the era of neural machine translation (NMT), we show how insights from word alignment inspired the attention mechanism fundamental to present-day NMT. The second part shifts to a survey approach. We cover neural word aligners, showing the slow but steady progress towards surpassing GIZA++ performance. Finally, we cover the present-day applications of word alignment, from cross-lingual annotation projection, to improving translation.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 30 of its 77 references that are in the map. First posted on arXiv 30 Nov 2022.
Builds on
30 of 77 references.
References · 30
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
- 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
- 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
- 2021Word Alignment by Fine-tuning Embeddings on Parallel Corporacited by 186
- 2020SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddingscited by 155
- 2020Neural CRF Model for Sentence Alignment in Text Simplificationcited by 106
- 2019Jointly Learning to Align and Translate with Transformer Modelscited by 75
- 2016Incorporating Discrete Translation Lexicons into Neural Machine Translationcited by 33
- 2020Accurate Word Alignment Induction from Neural Machine Translationcited by 32
18 more
- 2016Neural Machine Translation with Supervised Attentioncited by 31
- 2017What does Attention in Neural Machine Translation Pay Attention to?cited by 30
- 2020End-to-End Neural Word Alignment Outperforms GIZA++cited by 28
- 2019Saliency-driven Word Alignment Interpretation for Neural Machine Translationcited by 27
- 2020A Supervised Word Alignment Method based on Cross-Language Span Prediction using Multilingual BERTcited by 27
- 2017Neural Machine Translation and Sequence-to-sequence Models: A Tutorialcited by 25
- 2018Neural Network Models for Paraphrase Identification, Semantic Textual Similarity, Natural Language Inference, and Question Answeringcited by 23
- 2020Mask-Align: Self-Supervised Neural Word Alignmentcited by 20
- 2018On The Alignment Problem In Multi-Head Attention-Based Neural Machine Translationcited by 17
- 2016Incorporating Structural Alignment Biases into an Attentional Neural Translation Modelcited by 16
- 2021Neural semi-Markov CRF for Monolingual Word Alignmentcited by 15
- 2019A Discriminative Neural Model for Cross-Lingual Word Alignmentcited by 14
- 2021Attention Weights in Transformer NMT Fail Aligning Words Between Sequences but Largely Explain Model Predictionscited by 11
- 2019Cross-Lingual Transfer of Semantic Roles: From Raw Text to Semantic Rolescited by 9
- 2016Cross-Lingual Syntactic Transfer with Limited Resourcescited by 8
- 2017Transferring Semantic Roles Using Translation and Syntactic Informationcited by 7
- 2016Vocabulary Manipulation for Neural Machine Translationcited by 5
- 2015Agreement-based Joint Training for Bidirectional Attention-based Neural Machine Translationcited by 4
Built on by
Citing papers per year: 2023: 1; 2024: 2.
3 papers.