# The Effect of Alignment Objectives on Code-Switching Translation

arXiv: https://arxiv.org/abs/2309.05044 · HTML: https://mlsanity.com/paper/2309.05044/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 60 references that are in the map.
- First posted on arXiv 10 Sep 2023.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [Convolutional Sequence to Sequence Learning](https://mlsanity.com/paper/1705.03122/) (2017)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation](https://mlsanity.com/paper/1611.04558/) (2016)
- [Additive Margin Softmax for Face Verification](https://mlsanity.com/paper/1801.05599/) (2018)
- [Contrastive Learning for Many-to-many Multilingual Neural Machine Translation](https://mlsanity.com/paper/2105.09501/) (2021)
- [GLUECoS : An Evaluation Benchmark for Code-Switched NLP](https://mlsanity.com/paper/2004.12376/) (2020)
- [LinCE: A Centralized Benchmark for Linguistic Code-switching Evaluation](https://mlsanity.com/paper/2005.04322/) (2020)
- [Training Neural Machine Translation To Apply Terminology Constraints](https://mlsanity.com/paper/1906.01105/) (2019)
- [Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information](https://mlsanity.com/paper/2010.03142/) (2020)
- [SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweets](https://mlsanity.com/paper/2008.04277/) (2020)
- [The Missing Ingredient in Zero-Shot Neural Machine Translation](https://mlsanity.com/paper/1903.07091/) (2019)
- [Improving Multilingual Sentence Embedding using Bi-directional Dual Encoder with Additive Margin Softmax](https://mlsanity.com/paper/1902.08564/) (2019)
- [Code-Switched Language Models Using Neural Based Synthetic Data from Parallel Sentences](https://mlsanity.com/paper/1909.08582/) (2019)
- [Code-Switching for Enhancing NMT with Pre-Specified Translation](https://mlsanity.com/paper/1904.09107/) (2019)
- [Explicit Alignment Objectives for Multilingual Bidirectional Encoders](https://mlsanity.com/paper/2010.07972/) (2020)
- [Code-switching pre-training for neural machine translation](https://mlsanity.com/paper/2009.08088/) (2020)
- [Effective Parallel Corpus Mining using Bilingual Sentence Embeddings](https://mlsanity.com/paper/1807.11906/) (2018)
- [Overview for the Second Shared Task on Language Identification in Code-Switched Data](https://mlsanity.com/paper/1909.13016/) (2019)

## Built on by (most cited first, 1 of 1)

- [Low-resource Machine Translation for Code-switched Kazakh-Russian Language Pair](https://mlsanity.com/paper/2503.20007/) (2025)

## Abstract (arXiv)

One of the things that need to change when it comes to machine translation is the models' ability to translate code-switching content, especially with the rise of social media and user-generated content. In this paper, we are proposing a way of training a single machine translation model that is able to translate monolingual sentences from one language to another, along with translating code-switched sentences to either language. This model can be considered a bilingual model in the human sense. For better use of parallel data, we generated synthetic code-switched (CSW) data along with an alignment loss on the encoder to align representations across languages. Using the WMT14 English-French (En-Fr) dataset, the trained model strongly outperforms bidirectional baselines on code-switched translation while maintaining quality for non-code-switched (monolingual) data.