A Multilingual Bag-of-Entities Model for Zero-Shot Cross-Lingual Text Classification
Not on the map. Why?
Abstract (arXiv)
We present a multilingual bag-of-entities model that effectively boosts the performance of zero-shot cross-lingual text classification by extending a multilingual pre-trained language model (e.g., M-BERT). It leverages the multilingual nature of Wikidata: entities in multiple languages representing the same concept are defined with a unique identifier. This enables entities described in multiple languages to be represented using shared embeddings. A model trained on entity features in a resource-rich language can thus be directly applied to other languages. Our experimental results on cross-lingual topic classification (using the MLDoc and TED-CLDC datasets) and entity typing (using the SHINRA2020-ML dataset) show that the proposed model consistently outperforms state-of-the-art models.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 22 of its 40 references that are in the map. First posted on arXiv 15 Oct 2021.
Builds on
22 of 40 references.
References · 22
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
- 2019Cross-lingual Language Model Pretrainingcited by 1,088
- 2016Complex Embeddings for Simple Link Predictioncited by 897
- 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
- 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
- 2019KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representationcited by 356
- 2020LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentioncited by 326
- 2019Knowledge Enhanced Contextual Word Representationscited by 294
- 2019Emerging Cross-lingual Structure in Pretrained Language Modelscited by 180
- 2020Entities as Experts: Sparse Memory Access with Entity Supervisioncited by 100
10 more
- 2019E-BERT: Efficient-Yet-Effective Entity Embeddings for BERTcited by 96
- 2018A Corpus for Multilingual Document Classification in Eight Languagescited by 42
- 2019MultiFiT: Efficient Multi-lingual Language Model Fine-tuningcited by 31
- 2019Adversarial Learning with Contextual Embeddings for Zero-resource Cross-lingual Classification and NERcited by 25
- 2019Enriching BERT with Knowledge Graph Embeddings for Document Classificationcited by 22
- 2014Multilingual Models for Compositional Distributed Semanticscited by 18
- 2021Injecting Knowledge Base Information into End-to-End Joint Entity and Relation Extraction and Coreference Resolutioncited by 12
- 2020Leveraging Adversarial Training in Self-Learning for Cross-Lingual Text Classificationcited by 11
- 2019Neural Attentive Bag-of-Entities Model for Text Classificationcited by 9
- 2019Bridging the domain gap in cross-lingual document classificationcited by 8
Built on by
Citing papers per year: 2022: 1; 2023: 0; 2024: 1.
2 papers.