# Analyzing Architectures for Neural Machine Translation Using Low Computational Resources

arXiv: https://arxiv.org/abs/2111.03813 · HTML: https://mlsanity.com/paper/2111.03813/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 6 Nov 2021.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Convolutional Sequence to Sequence Learning](https://mlsanity.com/paper/1705.03122/) (2017)
- [Six Challenges for Neural Machine Translation](https://mlsanity.com/paper/1706.03872/) (2017)
- [WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia](https://mlsanity.com/paper/1907.05791/) (2019)
- [Facebook FAIR's WMT19 News Translation Task Submission](https://mlsanity.com/paper/1907.06616/) (2019)
- [Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages](https://mlsanity.com/paper/2104.05596/) (2021)
- [Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism](https://mlsanity.com/paper/1601.01073/) (2016)
- [The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation](https://mlsanity.com/paper/1804.09849/) (2018)
- [TICO-19: the Translation Initiative for Covid-19](https://mlsanity.com/paper/2007.01788/) (2020)
- [A Convolutional Encoder Model for Neural Machine Translation](https://mlsanity.com/paper/1611.02344/) (2016)
- [PMIndia -- A Collection of Parallel Corpora of Languages of India](https://mlsanity.com/paper/2001.09907/) (2020)
- [Complete Multilingual Neural Machine Translation](https://mlsanity.com/paper/2010.10239/) (2020)
- [SYSTRAN's Pure Neural Machine Translation Systems](https://mlsanity.com/paper/1610.05540/) (2016)
- [Revisiting Low Resource Status of Indian Languages in Machine Translation](https://mlsanity.com/paper/2008.04860/) (2020)
- [Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translation](https://mlsanity.com/paper/1606.04199/) (2016)
- [Marathi To English Neural Machine Translation With Near Perfect Corpus And Transformers](https://mlsanity.com/paper/2002.11643/) (2020)

## Abstract (arXiv)

With the recent developments in the field of Natural Language Processing, there has been a rise in the use of different architectures for Neural Machine Translation. Transformer architectures are used to achieve state-of-the-art accuracy, but they are very computationally expensive to train. Everyone cannot have such setups consisting of high-end GPUs and other resources. We train our models on low computational resources and investigate the results. As expected, transformers outperformed other architectures, but there were some surprising results. Transformers consisting of more encoders and decoders took more time to train but had fewer BLEU scores. LSTM performed well in the experiment and took comparatively less time to train than transformers, making it suitable to use in situations having time constraints.