# Incorporating Structural Alignment Biases into an Attentional Neural Translation Model

arXiv: https://arxiv.org/abs/1601.01085 · HTML: https://mlsanity.com/paper/1601.01085/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jan 2016.

## Built on by (most cited first, 16 of 16)

- [Word Alignment by Fine-tuning Embeddings on Parallel Corpora](https://mlsanity.com/paper/2101.08231/) (2021)
- [Sequence-to-Sequence Learning with Latent Neural Grammars](https://mlsanity.com/paper/2109.01135/) (2021)
- [Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMT](https://mlsanity.com/paper/2109.01396/) (2021)
- [Improving Neural Machine Translation by Bidirectional Training](https://mlsanity.com/paper/2109.07780/) (2021)
- [Neural OCR Post-Hoc Correction of Historical Corpora](https://mlsanity.com/paper/2102.00583/) (2021)
- [Lexically Aware Semi-Supervised Learning for OCR Post-Correction](https://mlsanity.com/paper/2111.02622/) (2021)
- [Inducing and Using Alignments for Transition-based AMR Parsing](https://mlsanity.com/paper/2205.01464/) (2022)
- [The USYD-JD Speech Translation System for IWSLT 2021](https://mlsanity.com/paper/2107.11572/) (2021)
- [Hierarchical Phrase-based Sequence-to-Sequence Learning](https://mlsanity.com/paper/2211.07906/) (2022)
- [Code to Comment Translation: A Comparative Study on Model Effectiveness & Errors](https://mlsanity.com/paper/2106.08415/) (2021)
- [Compositional Generalisation with Structured Reordering and Fertility Layers](https://mlsanity.com/paper/2210.03183/) (2022)
- [On Biasing Transformer Attention Towards Monotonicity](https://mlsanity.com/paper/2104.03945/) (2021)
- [Word Alignment in the Era of Deep Learning: A Tutorial](https://mlsanity.com/paper/2212.00138/) (2022)
- [DAMP: Doubly Aligned Multilingual Parser for Task-Oriented Dialogue](https://mlsanity.com/paper/2212.08054/) (2022)
- [Noisy Parallel Data Alignment](https://mlsanity.com/paper/2301.09685/) (2023)
- [An empirical analysis of phrase-based and neural machine translation](https://mlsanity.com/paper/2103.03108/) (2021)

## Abstract (arXiv)

Neural encoder-decoder models of machine translation have achieved impressive results, rivalling traditional translation models. However their modelling formulation is overly simplistic, and omits several key inductive biases built into traditional models. In this paper we extend the attentional neural translation model to include structural biases from word based alignment models, including positional bias, Markov conditioning, fertility and agreement over translation directions. We show improvements over a baseline attentional model and standard phrase-based model over several language pairs, evaluating on difficult languages in a low resource setting.