# Multilingual Code-Switching for Zero-Shot Cross-Lingual Intent Prediction and Slot Filling

arXiv: https://arxiv.org/abs/2103.07792 · HTML: https://mlsanity.com/paper/2103.07792/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 71 references that are in the map.
- First posted on arXiv 13 Mar 2021.

## Builds on (most cited first, 30 of 34)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [XNLI: Evaluating Cross-lingual Sentence Representations](https://mlsanity.com/paper/1809.05053/) (2018)
- [How multilingual is Multilingual BERT?](https://mlsanity.com/paper/1906.01502/) (2019)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [Zero-Shot Learning -- The Good, the Bad and the Ugly](https://mlsanity.com/paper/1703.04394/) (2017)
- [BERT for Joint Intent Classification and Slot Filling](https://mlsanity.com/paper/1902.10909/) (2019)
- [Emerging Cross-lingual Structure in Pretrained Language Models](https://mlsanity.com/paper/1911.01464/) (2019)
- [Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/1609.01454/) (2016)
- [Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialog](https://mlsanity.com/paper/1810.13327/) (2018)
- [A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/1907.00390/) (2019)
- [End-to-End Slot Alignment and Recognition for Cross-Lingual NLU](https://mlsanity.com/paper/2004.14353/) (2020)
- [CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLP](https://mlsanity.com/paper/2006.06402/) (2020)
- [Predicting Performance for Natural Language Processing Tasks](https://mlsanity.com/paper/2005.00870/) (2020)
- [Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification](https://mlsanity.com/paper/1606.01614/) (2016)
- [Neural Cross-Lingual Named Entity Recognition with Minimal Resources](https://mlsanity.com/paper/1808.09861/) (2018)
- [Weakly Supervised Cross-Lingual Named Entity Recognition via Effective Annotation and Representation Projection](https://mlsanity.com/paper/1707.02483/) (2017)
- [Towards Sub-Word Level Compositions for Sentiment Analysis of Hindi-English Code Mixed Text](https://mlsanity.com/paper/1611.00472/) (2016)
- [Cross-lingual Zero- and Few-shot Hate Speech Detection Utilising Frozen Transformer Language Models and AXEL](https://mlsanity.com/paper/2004.13850/) (2020)
- [DICT-MLM: Improved Multilingual Pre-Training using Bilingual Dictionaries](https://mlsanity.com/paper/2010.12566/) (2020)
- [Zero-Shot Cross-lingual Classification Using Multilingual Neural Machine Translation](https://mlsanity.com/paper/1809.04686/) (2018)
- [From English to Code-Switching: Transfer Learning with Strong Morphological Clues](https://mlsanity.com/paper/1909.05158/) (2019)
- [Subword-Level Language Identification for Intra-Word Code-Switching](https://mlsanity.com/paper/1904.01989/) (2019)
- [Distant Supervision from Disparate Sources for Low-Resource Part-of-Speech Tagging](https://mlsanity.com/paper/1808.09733/) (2018)

## Built on by (most cited first, 25 of 25)

- [LLaMA Beyond English: An Empirical Study on Language Capability Transfer](https://mlsanity.com/paper/2401.01055/) (2024)
- [Cross-Lingual Dialogue Dataset Creation via Outline-Based Generation](https://mlsanity.com/paper/2201.13405/) (2022)
- [Zero-Shot Cross-lingual Semantic Parsing](https://mlsanity.com/paper/2104.07554/) (2021)
- [Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.08570/) (2021)
- [Transforming Sequence Tagging Into A Seq2Seq Task](https://mlsanity.com/paper/2203.08378/) (2022)
- [Contextual Label Projection for Cross-Lingual Structured Prediction](https://mlsanity.com/paper/2309.08943/) (2023)
- [EntityCS: Improving Zero-Shot Cross-lingual Transfer with Entity-Centric Code Switching](https://mlsanity.com/paper/2210.12540/) (2022)
- [Multilingual Relation Classification via Efficient and Effective Prompting](https://mlsanity.com/paper/2210.13838/) (2022)
- [SentMix-3L: A Bangla-English-Hindi Code-Mixed Dataset for Sentiment Analysis](https://mlsanity.com/paper/2310.18023/) (2023)
- [Code-Mixed Probes Show How Pre-Trained Models Generalise On Code-Switched Text](https://mlsanity.com/paper/2403.04872/) (2024)
- [Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalities](https://mlsanity.com/paper/2510.07037/) (2025)
- [Transfer-Free Data-Efficient Multilingual Slot Labeling](https://mlsanity.com/paper/2305.13528/) (2023)
- [Mixed-Distil-BERT: Code-mixed Language Modeling for Bangla, English, and Hindi](https://mlsanity.com/paper/2309.10272/) (2023)
- [Parallel Corpora for Machine Translation in Low-resource Indic Languages: A Comprehensive Review](https://mlsanity.com/paper/2503.04797/) (2025)
- [Language Agnostic Code-Mixing Data Augmentation by Predicting Linguistic Patterns](https://mlsanity.com/paper/2211.07628/) (2022)
- [Self-Augmentation Improves Zero-Shot Cross-Lingual Transfer](https://mlsanity.com/paper/2309.10891/) (2023)
- [Call Larisa Ivanovna: Code-Switching Fools Multilingual NLU Models](https://mlsanity.com/paper/2109.14350/) (2021)
- [NATURE: Natural Auxiliary Text Utterances for Realistic Spoken Language Evaluation](https://mlsanity.com/paper/2111.05196/) (2021)
- [Multilingual Multimodality: A Taxonomical Survey of Datasets, Techniques, Challenges and Opportunities](https://mlsanity.com/paper/2210.16960/) (2022)
- [Automatic Identification of Motivation for Code-Switching in Speech Transcripts](https://mlsanity.com/paper/2212.08565/) (2022)
- [Improving Cross-lingual Representation for Semantic Retrieval with Code-switching](https://mlsanity.com/paper/2403.01364/) (2024)
- [Unraveling Code-Mixing Patterns in Migration Discourse: Automated Detection and Analysis of Online Conversations on Reddit](https://mlsanity.com/paper/2406.08633/) (2024)
- [ViWOZ: A Multi-Domain Task-Oriented Dialogue Systems Dataset For Low-resource Language](https://mlsanity.com/paper/2203.07742/) (2022)
- [WolBanking77: Wolof Banking Speech Intent Classification Dataset](https://mlsanity.com/paper/2509.19271/) (2025)
- [Debiasing Large Language Models toward Social Factors in Online Behavior Analytics through Prompt Knowledge Tuning](https://mlsanity.com/paper/2603.27057/) (2026)

## Abstract (arXiv)

Predicting user intent and detecting the corresponding slots from text are two key problems in Natural Language Understanding (NLU). In the context of zero-shot learning, this task is typically approached by either using representations from pre-trained multilingual transformers such as mBERT, or by machine translating the source data into the known target language and then fine-tuning. Our work focuses on a particular scenario where the target language is unknown during training. To this goal, we propose a novel method to augment the monolingual source data using multilingual code-switching via random translations to enhance a transformer's language neutrality when fine-tuning it for a downstream task. This method also helps discover novel insights on how code-switching with different language families around the world impact the performance on the target language. Experiments on the benchmark dataset of MultiATIS++ yielded an average improvement of +4.2% in accuracy for intent task and +1.8% in F1 for slot task using our method over the state-of-the-art across 8 different languages. Furthermore, we present an application of our method for crisis informatics using a new human-annotated tweet dataset of slot filling in English and Haitian Creole, collected during Haiti earthquake disaster.