# Cross-lingual Text Classification with Heterogeneous Graph Neural Network

arXiv: https://arxiv.org/abs/2105.11246 · HTML: https://mlsanity.com/paper/2105.11246/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 37 references that are in the map.
- First posted on arXiv 24 May 2021.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization](https://mlsanity.com/paper/2003.11080/) (2020)
- [Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/1809.05679/) (2018)
- [Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT](https://mlsanity.com/paper/1904.09077/) (2019)
- [XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation](https://mlsanity.com/paper/2004.01401/) (2020)
- [Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialog](https://mlsanity.com/paper/1810.13327/) (2018)
- [Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks](https://mlsanity.com/paper/1909.00964/) (2019)
- [CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLP](https://mlsanity.com/paper/2006.06402/) (2020)
- [Every Document Owns Its Structure: Inductive Text Classification via Graph Neural Networks](https://mlsanity.com/paper/2004.13826/) (2020)
- [Be More with Less: Hypergraph Attention Networks for Inductive Text Classification](https://mlsanity.com/paper/2011.00387/) (2020)
- [SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysis](https://mlsanity.com/paper/2005.05635/) (2020)
- [Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification](https://mlsanity.com/paper/1606.01614/) (2016)
- [Multi-Scale Self-Attention for Text Classification](https://mlsanity.com/paper/1912.00544/) (2019)
- [Cross-lingual Distillation for Text Classification](https://mlsanity.com/paper/1705.02073/) (2017)
- [Exploiting Cross-Lingual Subword Similarities in Low-Resource Document Classification](https://mlsanity.com/paper/1812.09617/) (2018)

## Built on by (most cited first, 12 of 12)

- [Getting the Most out of Simile Recognition](https://mlsanity.com/paper/2211.05984/) (2022)
- [Generalized Funnelling: Ensemble Learning and Heterogeneous Document Embeddings for Cross-Lingual Text Classification](https://mlsanity.com/paper/2110.14764/) (2021)
- [Measuring Catastrophic Forgetting in Cross-Lingual Transfer Paradigms: Exploring Tuning Strategies](https://mlsanity.com/paper/2309.06089/) (2023)
- [Incorporating Lexical and Syntactic Knowledge for Unsupervised Cross-Lingual Transfer](https://mlsanity.com/paper/2404.16627/) (2024)
- [Decoding Climate Disagreement: A Graph Neural Network-Based Approach to Understanding Social Media Dynamics](https://mlsanity.com/paper/2407.07038/) (2024)
- [Text Classification using Graph Convolutional Networks: A Comprehensive Survey](https://mlsanity.com/paper/2410.09399/) (2024)
- [FewShotTextGCN: K-hop neighborhood regularization for few-shot learning on graphs](https://mlsanity.com/paper/2301.10481/) (2023)
- [Graph Neural Networks for Antisocial Behavior Detection on Twitter](https://mlsanity.com/paper/2312.16755/) (2023)
- [Efficient Multi-domain Text Recognition Deep Neural Network Parameterization with Residual Adapters](https://mlsanity.com/paper/2401.00971/) (2024)
- [Scalable Multi-Task Learning for Particle Collision Event Reconstruction with Heterogeneous Graph Neural Networks](https://mlsanity.com/paper/2504.21844/) (2025)
- [BhashaSetu: Cross-Lingual Knowledge Transfer from High-Resource to Extreme Low-Resource Languages](https://mlsanity.com/paper/2602.05599/) (2026)
- [Domain-Agnostic Neural Topic Modeling with Contextual Token-Level Semantic Graph Representation](https://mlsanity.com/paper/2608.16269/) (2026)

## Abstract (arXiv)

Cross-lingual text classification aims at training a classifier on the source language and transferring the knowledge to target languages, which is very useful for low-resource languages. Recent multilingual pretrained language models (mPLM) achieve impressive results in cross-lingual classification tasks, but rarely consider factors beyond semantic similarity, causing performance degradation between some language pairs. In this paper we propose a simple yet effective method to incorporate heterogeneous information within and across languages for cross-lingual text classification using graph convolutional networks (GCN). In particular, we construct a heterogeneous graph by treating documents and words as nodes, and linking nodes with different relations, which include part-of-speech roles, semantic similarity, and document translations. Extensive experiments show that our graph-based method significantly outperforms state-of-the-art models on all tasks, and also achieves consistent performance gain over baselines in low-resource settings where external tools like translators are unavailable.