# Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialog

arXiv: https://arxiv.org/abs/1810.13327 · HTML: https://mlsanity.com/paper/1810.13327/

## Key facts

- Cited by 125 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2018.

## Built on by (most cited first, 30 of 125)

- [MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages](https://mlsanity.com/paper/2204.08582/) (2022)
- [An Empirical Survey of Data Augmentation for Limited Data Learning in NLP](https://mlsanity.com/paper/2106.07499/) (2021)
- [Language Models are Few-shot Multilingual Learners](https://mlsanity.com/paper/2109.07684/) (2021)
- [Revisiting Mahalanobis Distance for Transformer-Based Out-of-Domain Detection](https://mlsanity.com/paper/2101.03778/) (2021)
- [BiToD: A Bilingual Multi-Domain Dataset For Task-Oriented Dialogue Modeling](https://mlsanity.com/paper/2106.02787/) (2021)
- [A Survey on Spoken Language Understanding: Recent Advances and New Frontiers](https://mlsanity.com/paper/2103.03095/) (2021)
- [Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers](https://mlsanity.com/paper/2404.04925/) (2024)
- [DuRecDial 2.0: A Bilingual Parallel Corpus for Conversational Recommendation](https://mlsanity.com/paper/2109.08877/) (2021)
- [Recent Advances of Foundation Language Models-based Continual Learning: A Survey](https://mlsanity.com/paper/2405.18653/) (2024)
- [From Masked Language Modeling to Translation: Non-English Auxiliary Tasks Improve Zero-shot Spoken Language Understanding](https://mlsanity.com/paper/2105.07316/) (2021)
- [Multilingual and Cross-Lingual Intent Detection from Spoken Data](https://mlsanity.com/paper/2104.08524/) (2021)
- [SPACE-3: Unified Dialog Model Pre-training for Task-Oriented Dialog Understanding and Generation](https://mlsanity.com/paper/2209.06664/) (2022)
- [AdaMergeX: Cross-Lingual Transfer with Large Language Models via Adaptive Adapter Merging](https://mlsanity.com/paper/2402.18913/) (2024)
- [Weakly Supervised Data Augmentation Through Prompting for Dialogue Understanding](https://mlsanity.com/paper/2210.14169/) (2022)
- [GlobalWoZ: Globalizing MultiWoZ to Develop Multilingual Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2110.07679/) (2021)
- [A Survey on Out-of-Distribution Detection in NLP](https://mlsanity.com/paper/2305.03236/) (2023)
- [SPACE-2: Tree-Structured Semi-Supervised Contrastive Pre-training for Task-Oriented Dialog Understanding](https://mlsanity.com/paper/2209.06638/) (2022)
- [Multilingual Code-Switching for Zero-Shot Cross-Lingual Intent Prediction and Slot Filling](https://mlsanity.com/paper/2103.07792/) (2021)
- [Cross-Lingual Dialogue Dataset Creation via Outline-Based Generation](https://mlsanity.com/paper/2201.13405/) (2022)
- [A Survey on Recent Advances in Conversational Data Generation](https://mlsanity.com/paper/2405.13003/) (2024)
- [STOP: A dataset for Spoken Task Oriented Semantic Parsing](https://mlsanity.com/paper/2207.10643/) (2022)
- [GOLD: Improving Out-of-Scope Detection in Dialogues using Data Augmentation](https://mlsanity.com/paper/2109.03079/) (2021)
- [Intent Classification and Slot Filling for Privacy Policies](https://mlsanity.com/paper/2101.00123/) (2021)
- [Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.08570/) (2021)
- [Translate & Fill: Improving Zero-Shot Multilingual Semantic Parsing with Synthetic Data](https://mlsanity.com/paper/2109.04319/) (2021)
- [Effects of diversity incentives on sample diversity and downstream model performance in LLM-based text augmentation](https://mlsanity.com/paper/2401.06643/) (2024)
- [Multi3WOZ: A Multilingual, Multi-Domain, Multi-Parallel Dataset for Training and Evaluating Culturally Adapted Task-Oriented Dialog Systems](https://mlsanity.com/paper/2307.14031/) (2023)
- [Transforming Sequence Tagging Into A Seq2Seq Task](https://mlsanity.com/paper/2203.08378/) (2022)
- [MULTI3NLU++: A Multilingual, Multi-Intent, Multi-Domain Dataset for Natural Language Understanding in Task-Oriented Dialogue](https://mlsanity.com/paper/2212.10455/) (2022)
- [Translation and Fusion Improves Zero-shot Cross-lingual Information Extraction](https://mlsanity.com/paper/2305.13582/) (2023)

## Abstract (arXiv)

One of the first steps in the utterance interpretation pipeline of many task-oriented conversational AI systems is to identify user intents and the corresponding slots. Since data collection for machine learning models for this task is time-consuming, it is desirable to make use of existing data in a high-resource language to train models in low-resource languages. However, development of such models has largely been hindered by the lack of multilingual training data. In this paper, we present a new data set of 57k annotated utterances in English (43k), Spanish (8.6k) and Thai (5k) across the domains weather, alarm, and reminder. We use this data set to evaluate three different cross-lingual transfer methods: (1) translating the training data, (2) using cross-lingual pre-trained embeddings, and (3) a novel method of using a multilingual machine translation encoder as contextual word representations. We find that given several hundred training examples in the the target language, the latter two methods outperform translating the training data. Further, in very low-resource settings, multilingual contextual word representations give better results than using cross-lingual static embeddings. We also compare the cross-lingual methods to using monolingual resources in the form of contextual ELMo representations and find that given just small amounts of target language data, this method outperforms all cross-lingual methods, which highlights the need for more sophisticated cross-lingual methods.