# Zero-Shot Cross-lingual Classification Using Multilingual Neural Machine Translation

arXiv: https://arxiv.org/abs/1809.04686 · HTML: https://mlsanity.com/paper/1809.04686/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Sep 2018.

## Built on by (most cited first, 19 of 19)

- [MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages](https://mlsanity.com/paper/2204.08582/) (2022)
- [mSLAM: Massively multilingual joint pre-training for speech and text](https://mlsanity.com/paper/2202.01374/) (2022)
- [Multilingual Code-Switching for Zero-Shot Cross-Lingual Intent Prediction and Slot Filling](https://mlsanity.com/paper/2103.07792/) (2021)
- [Translate & Fill: Improving Zero-Shot Multilingual Semantic Parsing with Synthetic Data](https://mlsanity.com/paper/2109.04319/) (2021)
- [Modelling Latent Translations for Cross-Lingual Transfer](https://mlsanity.com/paper/2107.11353/) (2021)
- [MergeDistill: Merging Pre-trained Language Models using Distillation](https://mlsanity.com/paper/2106.02834/) (2021)
- [Code-switched inspired losses for generic spoken dialog representations](https://mlsanity.com/paper/2108.12465/) (2021)
- [Are the Multilingual Models Better? Improving Czech Sentiment with Transformers](https://mlsanity.com/paper/2108.10640/) (2021)
- [Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documents](https://mlsanity.com/paper/2109.10341/) (2021)
- [Can Machine Translation Bridge Multilingual Pretraining and Cross-lingual Transfer Learning?](https://mlsanity.com/paper/2403.16777/) (2024)
- [Soft Layer Selection with Meta-Learning for Zero-Shot Cross-Lingual Transfer](https://mlsanity.com/paper/2107.09840/) (2021)
- [Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representation](https://mlsanity.com/paper/2210.09773/) (2022)
- [The Interpreter Understands Your Meaning: End-to-end Spoken Language Understanding Aided by Speech Translation](https://mlsanity.com/paper/2305.09652/) (2023)
- [Parallel-SFT: Improving Zero-Shot Cross-Programming-Language Transfer for Code RL](https://mlsanity.com/paper/2604.20835/) (2026)
- [Improving Cross-Lingual Reading Comprehension with Self-Training](https://mlsanity.com/paper/2105.03627/) (2021)
- [Out-of-Distribution Generalization in Text Classification: Past, Present, and Future](https://mlsanity.com/paper/2305.14104/) (2023)
- [Cross-lingual Aspect-Based Sentiment Analysis: A Survey on Tasks, Approaches, and Challenges](https://mlsanity.com/paper/2508.09516/) (2025)
- [Zero-Shot Cross-Lingual Transfer in Legal Domain Using Transformer Models](https://mlsanity.com/paper/2111.14192/) (2021)
- [tmn at SemEval-2023 Task 9: Multilingual Tweet Intimacy Detection using XLM-T, Google Translate, and Ensemble Learning](https://mlsanity.com/paper/2304.04054/) (2023)

## Abstract (arXiv)

Transferring representations from large supervised tasks to downstream tasks has shown promising results in AI fields such as Computer Vision and Natural Language Processing (NLP). In parallel, the recent progress in Machine Translation (MT) has enabled one to train multilingual Neural MT (NMT) systems that can translate between multiple languages and are also capable of performing zero-shot translation. However, little attention has been paid to leveraging representations learned by a multilingual NMT system to enable zero-shot multilinguality in other NLP tasks. In this paper, we demonstrate a simple framework, a multilingual Encoder-Classifier, for cross-lingual transfer learning by reusing the encoder from a multilingual NMT system and stitching it with a task-specific classifier component. Our proposed model achieves significant improvements in the English setup on three benchmark tasks - Amazon Reviews, SST and SNLI. Further, our system can perform classification in a new language for which no classification data was seen during training, showing that zero-shot classification is possible and remarkably competitive. In order to understand the underlying factors contributing to this finding, we conducted a series of analyses on the effect of the shared vocabulary, the training data type for NMT, classifier complexity, encoder representation power, and model generalization on zero-shot performance. Our results provide strong evidence that the representations learned from multilingual NMT systems are widely applicable across languages and tasks.