# Cross-Lingual Text Classification with Minimal Resources by Transferring a Sparse Teacher

arXiv: https://arxiv.org/abs/2010.02562 · HTML: https://mlsanity.com/paper/2010.02562/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Oct 2020.

## Built on by (most cited first, 7 of 7)

- [Building Machine Translation Systems for the Next Thousand Languages](https://mlsanity.com/paper/2205.03983/) (2022)
- [Intent Identification and Entity Extraction for Healthcare Queries in Indic Languages](https://mlsanity.com/paper/2302.09685/) (2023)
- [Cross-Lingual Knowledge Distillation for Answer Sentence Selection in Low-Resource Languages](https://mlsanity.com/paper/2305.16302/) (2023)
- [Universal Cross-Lingual Text Classification](https://mlsanity.com/paper/2406.11028/) (2024)
- [Generalized Funnelling: Ensemble Learning and Heterogeneous Document Embeddings for Cross-Lingual Text Classification](https://mlsanity.com/paper/2110.14764/) (2021)
- [Cross-lingual Transfer for Text Classification with Dictionary-based Heterogeneous Graph](https://mlsanity.com/paper/2109.04400/) (2021)
- [ADMEDTAGGER: an annotation framework for distillation of expert knowledge for the Polish medical language](https://mlsanity.com/paper/2601.09722/) (2025)

## Abstract (arXiv)

Cross-lingual text classification alleviates the need for manually labeled documents in a target language by leveraging labeled documents from other languages. Existing approaches for transferring supervision across languages require expensive cross-lingual resources, such as parallel corpora, while less expensive cross-lingual representation learning approaches train classifiers without target labeled documents. In this work, we propose a cross-lingual teacher-student method, CLTS, that generates "weak" supervision in the target language using minimal cross-lingual resources, in the form of a small number of word translations. Given a limited translation budget, CLTS extracts and transfers only the most important task-specific seed words across languages and initializes a teacher classifier based on the translated seed words. Then, CLTS iteratively trains a more powerful student that also exploits the context of the seed words in unlabeled target documents and outperforms the teacher. CLTS is simple and surprisingly effective in 18 diverse languages: by transferring just 20 seed words, even a bag-of-words logistic regression student outperforms state-of-the-art cross-lingual methods (e.g., based on multilingual BERT). Moreover, CLTS can accommodate any type of student classifier: leveraging a monolingual BERT student leads to further improvements and outperforms even more expensive approaches by up to 12% in accuracy. Finally, CLTS addresses emerging tasks in low-resource languages using just a small number of word translations.