# From English to Code-Switching: Transfer Learning with Strong Morphological Clues

arXiv: https://arxiv.org/abs/1909.05158 · HTML: https://mlsanity.com/paper/1909.05158/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Sep 2019.

## Built on by (most cited first, 18 of 18)

- [The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges](https://mlsanity.com/paper/2212.09660/) (2022)
- [Multilingual Code-Switching for Zero-Shot Cross-Lingual Intent Prediction and Slot Filling](https://mlsanity.com/paper/2103.07792/) (2021)
- [Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots](https://mlsanity.com/paper/2103.09593/) (2021)
- [Can You Traducir This? Machine Translation for Code-Switched Input](https://mlsanity.com/paper/2105.04846/) (2021)
- [CALCS 2021 Shared Task: Machine Translation for Code-Switched Data](https://mlsanity.com/paper/2202.09625/) (2022)
- [CodemixedNLP: An Extensible and Open NLP Toolkit for Code-Mixing](https://mlsanity.com/paper/2106.06004/) (2021)
- [Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switching](https://mlsanity.com/paper/2203.08979/) (2022)
- [HIT: A Hierarchically Fused Deep Attention Network for Robust Code-mixed Language Representation](https://mlsanity.com/paper/2105.14600/) (2021)
- [Switch Point biased Self-Training: Re-purposing Pretrained Models for Code-Switching](https://mlsanity.com/paper/2111.01231/) (2021)
- [Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalities](https://mlsanity.com/paper/2510.07037/) (2025)
- [Low-resource Machine Translation for Code-switched Kazakh-Russian Language Pair](https://mlsanity.com/paper/2503.20007/) (2025)
- [A Comprehensive Understanding of Code-mixed Language Semantics using Hierarchical Transformer](https://mlsanity.com/paper/2204.12753/) (2022)
- [Progressive Sentiment Analysis for Code-Switched Text Data](https://mlsanity.com/paper/2210.14380/) (2022)
- [Automatic Identification of Motivation for Code-Switching in Speech Transcripts](https://mlsanity.com/paper/2212.08565/) (2022)
- [Unraveling Code-Mixing Patterns in Migration Discourse: Automated Detection and Analysis of Online Conversations on Reddit](https://mlsanity.com/paper/2406.08633/) (2024)
- [Adapting Multilingual Models to Code-Mixed Tasks via Model Merging](https://mlsanity.com/paper/2510.19782/) (2025)
- [PingPong: A Natural Benchmark for Multi-Turn Code-Switching Dialogues](https://mlsanity.com/paper/2601.17277/) (2026)
- [Detecting Anchors' Opinion in Hinghlish News Delivery](https://mlsanity.com/paper/2204.02155/) (2022)

## Abstract (arXiv)

Linguistic Code-switching (CS) is still an understudied phenomenon in natural language processing. The NLP community has mostly focused on monolingual and multi-lingual scenarios, but little attention has been given to CS in particular. This is partly because of the lack of resources and annotated data, despite its increasing occurrence in social media platforms. In this paper, we aim at adapting monolingual models to code-switched text in various tasks. Specifically, we transfer English knowledge from a pre-trained ELMo model to different code-switched language pairs (i.e., Nepali-English, Spanish-English, and Hindi-English) using the task of language identification. Our method, CS-ELMo, is an extension of ELMo with a simple yet effective position-aware attention mechanism inside its character convolutions. We show the effectiveness of this transfer learning step by outperforming multilingual BERT and homologous CS-unaware ELMo models and establishing a new state of the art in CS tasks, such as NER and POS tagging. Our technique can be expanded to more English-paired code-switched languages, providing more resources to the CS community.