Can Latent Alignments Improve Autoregressive Machine Translation?
Not on the map. Why?
Abstract (arXiv)
Latent alignment objectives such as CTC and AXE significantly improve non-autoregressive machine translation models. Can they improve autoregressive models as well? We explore the possibility of training autoregressive machine translation models with latent alignment objectives, and observe that, in practice, this approach results in degenerate models. We provide a theoretical explanation for these empirical results, and prove that latent alignment objectives are incompatible with teacher forcing.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 10 of its 14 references that are in the map. First posted on arXiv 19 Apr 2021.
Builds on
10 of 14 references.
References · 10
- 2017Attention Is All You Needcited by 59,932
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
- 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
- 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
- 2016Professor Forcing: A New Algorithm for Training Recurrent Networkscited by 214
- 2020Non-Autoregressive Machine Translation with Latent Alignmentscited by 96
- 2018End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classificationcited by 72
- 2020Aligned Cross Entropy for Non-Autoregressive Machine Translationcited by 65
- 2020Semi-Autoregressive Training Improves Mask-Predict Decodingcited by 33
Built on by
Citing papers per year: 2021: 1; 2022: 2.
3 papers.