# Temporal Attention Model for Neural Machine Translation

arXiv: https://arxiv.org/abs/1608.02927 · HTML: https://mlsanity.com/paper/1608.02927/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Aug 2016.

## Built on by (most cited first, 2 of 2)

- [Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videos](https://mlsanity.com/paper/2109.03223/) (2021)
- [Predicting Outcomes in Long COVID Patients with Spatiotemporal Attention](https://mlsanity.com/paper/2307.04770/) (2023)

## Abstract (arXiv)

Attention-based Neural Machine Translation (NMT) models suffer from attention deficiency issues as has been observed in recent research. We propose a novel mechanism to address some of these limitations and improve the NMT attention. Specifically, our approach memorizes the alignments temporally (within each sentence) and modulates the attention with the accumulated temporal memory, as the decoder generates the candidate translation. We compare our approach against the baseline NMT model and two other related approaches that address this issue either explicitly or implicitly. Large-scale experiments on two language pairs show that our approach achieves better and robust gains over the baseline and related NMT approaches. Our model further outperforms strong SMT baselines in some settings even without using ensembles.