# Parallel Attention Mechanisms in Neural Machine Translation

arXiv: https://arxiv.org/abs/1810.12427 · HTML: https://mlsanity.com/paper/1810.12427/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2018.

## Built on by (most cited first, 4 of 4)

- [Fine-Grained Address Segmentation for Attention-Based Variable-Degree Prefetching](https://mlsanity.com/paper/2205.02269/) (2022)
- [Enhanced Coherence-Aware Network with Hierarchical Disentanglement for Aspect-Category Sentiment Analysis](https://mlsanity.com/paper/2403.10214/) (2024)
- [NL2CMD: An Updated Workflow for Natural Language to Bash Commands Translation](https://mlsanity.com/paper/2302.07845/) (2023)
- [Lerna: Transformer Architectures for Configuring Error Correction Tools for Short- and Long-Read Genome Sequencing](https://mlsanity.com/paper/2112.10068/) (2021)

## Abstract (arXiv)

Recent papers in neural machine translation have proposed the strict use of attention mechanisms over previous standards such as recurrent and convolutional neural networks (RNNs and CNNs). We propose that by running traditionally stacked encoding branches from encoder-decoder attention- focused architectures in parallel, that even more sequential operations can be removed from the model and thereby decrease training time. In particular, we modify the recently published attention-based architecture called Transformer by Google, by replacing sequential attention modules with parallel ones, reducing the amount of training time and substantially improving BLEU scores at the same time. Experiments over the English to German and English to French translation tasks show that our model establishes a new state of the art.