mlsanityData to 14 Oct 2026

Priberam Labs at the NTCIR-15 SHINRA2020-ML: Classification Task

arXiv 2105.05605 ↗First posted 12 May 2021cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Wikipedia is an online encyclopedia available in 285 languages. It composes an extremely relevant Knowledge Base (KB), which could be leveraged by automatic systems for several purposes. However, the structure and organisation of such information are not prone to automatic parsing and understanding and it is, therefore, necessary to structure this knowledge. The goal of the current SHINRA2020-ML task is to leverage Wikipedia pages in order to categorise their corresponding entities across 268 hierarchical categories, belonging to the Extended Named Entity (ENE) ontology. In this work, we propose three distinct models based on the contextualised embeddings yielded by Multilingual BERT. We explore the performances of a linear layer with and without explicit usage of the ontology's hierarchy, and a Gated Recurrent Units (GRU) layer. We also test several pooling strategies to leverage BERT's embeddings and selection criteria based on the labels' scores. We were able to achieve good performance across a large variety of languages, including those not seen during the fine-tuning process (zero-shot languages).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 21 references that are in the map. First posted on arXiv 12 May 2021.

Builds on

13 of 21 references.

References · 13

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  4. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  5. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  6. 2019How multilingual is Multilingual BERT?cited by 645
  7. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  8. 2017A Survey Of Cross-lingual Word Embedding Modelscited by 143
  9. 2020A Comparison of LSTM and BERT for Small Corpuscited by 40
  10. 2019MultiFiT: Efficient Multi-lingual Language Model Fine-tuningcited by 31
  11. 2013Polyglot: Distributed Word Representations for Multilingual NLPcited by 19
  12. 2018Unsupervised Multilingual Word Embeddingscited by 16
1 more
  1. 2020Light-Weighted CNN for Text Classificationcited by 1

Built on by

Not cited yet.