# Domain Curricula for Code-Switched MT at MixMT 2022

arXiv: https://arxiv.org/abs/2210.17463 · HTML: https://mlsanity.com/paper/2210.17463/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 40 references that are in the map.
- First posted on arXiv 31 Oct 2022.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Pay Less Attention with Lightweight and Dynamic Convolutions](https://mlsanity.com/paper/1901.10430/) (2019)
- [The IIT Bombay English-Hindi Parallel Corpus](https://mlsanity.com/paper/1710.02855/) (2017)
- [A Survey of Code-switched Speech and Language Processing](https://mlsanity.com/paper/1904.00784/) (2019)
- [Balancing Training for Multilingual Neural Machine Translation](https://mlsanity.com/paper/2004.06748/) (2020)
- [The Missing Ingredient in Zero-Shot Neural Machine Translation](https://mlsanity.com/paper/1903.07091/) (2019)
- [Code-Switching for Enhancing NMT with Pre-Specified Translation](https://mlsanity.com/paper/1904.09107/) (2019)
- [Pretraining boosts out-of-domain robustness for pose estimation](https://mlsanity.com/paper/1909.11229/) (2019)
- [PHINC: A Parallel Hinglish Social Media Code-Mixed Corpus for Machine Translation](https://mlsanity.com/paper/2004.09447/) (2020)
- [Dynamic Data Selection for Neural Machine Translation](https://mlsanity.com/paper/1708.00712/) (2017)
- [HinGE: A Dataset for Generation and Evaluation of Code-Mixed Hinglish Text](https://mlsanity.com/paper/2107.03760/) (2021)
- [Can You Traducir This? Machine Translation for Code-Switched Input](https://mlsanity.com/paper/2105.04846/) (2021)
- [BilBOWA: Fast Bilingual Distributed Representations without Word Alignments](https://mlsanity.com/paper/1410.2455/) (2014)
- [Domain Differential Adaptation for Neural Machine Translation](https://mlsanity.com/paper/1910.02555/) (2019)
- [Trans-gram, Fast Cross-lingual Word-embeddings](https://mlsanity.com/paper/1601.02502/) (2016)
- [DaLC: Domain Adaptation Learning Curve Prediction for Neural Machine Translation](https://mlsanity.com/paper/2204.09259/) (2022)

## Abstract (arXiv)

In multilingual colloquial settings, it is a habitual occurrence to compose expressions of text or speech containing tokens or phrases of different languages, a phenomenon popularly known as code-switching or code-mixing (CMX). We present our approach and results for the Code-mixed Machine Translation (MixMT) shared task at WMT 2022: the task consists of two subtasks, monolingual to code-mixed machine translation (Subtask-1) and code-mixed to monolingual machine translation (Subtask-2). Most non-synthetic code-mixed data are from social media but gathering a significant amount of this kind of data would be laborious and this form of data has more writing variation than other domains, so for both subtasks, we experimented with data schedules for out-of-domain data. We jointly learn multiple domains of text by pretraining and fine-tuning, combined with a sentence alignment objective. We found that switching between domains caused improved performance in the domains seen earliest during training, but depleted the performance on the remaining domains. A continuous training run with strategically dispensed data of different domains showed a significantly improved performance over fine-tuning.