# Attentive fine-tuning of Transformers for Translation of low-resourced languages @LoResMT 2021

arXiv: https://arxiv.org/abs/2108.08556 · HTML: https://mlsanity.com/paper/2108.08556/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 58 references that are in the map.
- First posted on arXiv 19 Aug 2021.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Mixed Precision Training](https://mlsanity.com/paper/1710.03740/) (2017)
- [Language-agnostic BERT Sentence Embedding](https://mlsanity.com/paper/2007.01852/) (2020)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation](https://mlsanity.com/paper/1611.04558/) (2016)
- [Massively Multilingual Neural Machine Translation](https://mlsanity.com/paper/1903.00089/) (2019)
- [Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages](https://mlsanity.com/paper/2104.05596/) (2021)
- [The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MT](https://mlsanity.com/paper/2010.06354/) (2020)
- [Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder](https://mlsanity.com/paper/1611.04798/) (2016)
- [Multi-Source Neural Translation](https://mlsanity.com/paper/1601.00710/) (2016)
- [Semi-Supervised Learning for Neural Machine Translation](https://mlsanity.com/paper/1606.04596/) (2016)
- [Multilingual NMT with a language-independent attention bridge](https://mlsanity.com/paper/1811.00498/) (2018)
- [Investigating Backtranslation in Neural Machine Translation](https://mlsanity.com/paper/1804.06189/) (2018)
- [Revisiting Low Resource Status of Indian Languages in Machine Translation](https://mlsanity.com/paper/2008.04860/) (2020)
- [Exploiting Out-of-Domain Parallel Data through Multilingual Transfer Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/1907.03060/) (2019)
- [Findings of the LoResMT 2021 Shared Task on COVID and Sign Language for Low-resource Languages](https://mlsanity.com/paper/2108.06598/) (2021)
- [IIITT@LT-EDI-EACL2021-Hope Speech Detection: There is always Hope in Transformers](https://mlsanity.com/paper/2104.09066/) (2021)
- [A Survey of Orthographic Information in Machine Translation](https://mlsanity.com/paper/2008.01391/) (2020)
- [UVCE-IIITT@DravidianLangTech-EACL2021: Tamil Troll Meme Classification: You need to Pay more Attention](https://mlsanity.com/paper/2104.09081/) (2021)
- [Marathi To English Neural Machine Translation With Near Perfect Corpus And Transformers](https://mlsanity.com/paper/2002.11643/) (2020)

## Built on by (most cited first, 4 of 4)

- [gaHealth: An English-Irish Bilingual Corpus of Health Data](https://mlsanity.com/paper/2403.03575/) (2024)
- [adaptNMT: an open-source, language-agnostic development environment for Neural Machine Translation](https://mlsanity.com/paper/2403.02367/) (2024)
- [IIITT@Dravidian-CodeMix-FIRE2021: Transliterate or translate? Sentiment analysis of code-mixed text in Dravidian languages](https://mlsanity.com/paper/2111.07906/) (2021)
- [Pegasus@Dravidian-CodeMix-HASOC2021: Analyzing Social Media Content for Detection of Offensive Text](https://mlsanity.com/paper/2111.09836/) (2021)

## Abstract (arXiv)

This paper reports the Machine Translation (MT) systems submitted by the IIITT team for the English->Marathi and English->Irish language pairs LoResMT 2021 shared task. The task focuses on getting exceptional translations for rather low-resourced languages like Irish and Marathi. We fine-tune IndicTrans, a pretrained multilingual NMT model for English->Marathi, using external parallel corpus as input for additional training. We have used a pretrained Helsinki-NLP Opus MT English->Irish model for the latter language pair. Our approaches yield relatively promising results on the BLEU metrics. Under the team name IIITT, our systems ranked 1, 1, and 2 in English->Marathi, Irish->English, and English->Irish, respectively.