# Out of Thin Air: Is Zero-Shot Cross-Lingual Keyword Detection Better Than Unsupervised?

arXiv: https://arxiv.org/abs/2202.06650 · HTML: https://mlsanity.com/paper/2202.06650/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 43 references that are in the map.
- First posted on arXiv 14 Feb 2022.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT](https://mlsanity.com/paper/1904.09077/) (2019)
- [Incorporating Copying Mechanism in Sequence-to-Sequence Learning](https://mlsanity.com/paper/1603.06393/) (2016)
- [Deep Keyphrase Generation](https://mlsanity.com/paper/1704.06879/) (2017)
- [Simple and Effective Zero-shot Cross-lingual Phoneme Recognition](https://mlsanity.com/paper/2109.11680/) (2021)
- [Simple Unsupervised Keyphrase Extraction using Sentence Embeddings](https://mlsanity.com/paper/1801.04470/) (2018)
- [Unsupervised Keyphrase Extraction with Multipartite Graphs](https://mlsanity.com/paper/1803.08721/) (2018)
- [KPTimes: A Large-Scale Dataset for Keyphrase Generation on News Documents](https://mlsanity.com/paper/1911.12559/) (2019)
- [Scientific Information Extraction with Semi-supervised Neural Tagging](https://mlsanity.com/paper/1708.06075/) (2017)
- [Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Model](https://mlsanity.com/paper/1909.09587/) (2019)
- [TNT-KID: Transformer-based Neural Tagger for Keyword Identification](https://mlsanity.com/paper/2003.09166/) (2020)
- [Keyphrase Extraction from Scholarly Articles as Sequence Labeling using Contextualized Embeddings](https://mlsanity.com/paper/1910.08840/) (2019)
- [RaKUn: Rank-based Keyword extraction via Unsupervised learning and Meta vertex aggregation](https://mlsanity.com/paper/1907.06458/) (2019)
- [Extending Neural Keyword Extraction with TF-IDF tagset matching](https://mlsanity.com/paper/2102.00472/) (2021)

## Built on by (most cited first, 7 of 7)

- [The Zeno's Paradox of `Low-Resource' Languages](https://mlsanity.com/paper/2410.20817/) (2024)
- [AHAM: Adapt, Help, Ask, Model -- Harvesting LLMs for literature mining](https://mlsanity.com/paper/2312.15784/) (2023)
- [AdaptKeyBERT: An Attention-Based approach towards Few-Shot & Zero-Shot Domain Adaptation of KeyBERT](https://mlsanity.com/paper/2211.07499/) (2022)
- [An evaluation of LLMs and Google Translate for translation of selected Indian languages via sentiment and semantic analyses](https://mlsanity.com/paper/2503.21393/) (2025)
- [Utilizing Cognitive Signals Generated during Human Reading to Enhance Keyphrase Extraction from Microblogs](https://mlsanity.com/paper/2606.26485/) (2026)
- [Measuring Catastrophic Forgetting in Cross-Lingual Transfer Paradigms: Exploring Tuning Strategies](https://mlsanity.com/paper/2309.06089/) (2023)
- [Evaluating and explaining training strategies for zero-shot cross-lingual news sentiment analysis](https://mlsanity.com/paper/2409.20054/) (2024)

## Abstract (arXiv)

Keyword extraction is the task of retrieving words that are essential to the content of a given document. Researchers proposed various approaches to tackle this problem. At the top-most level, approaches are divided into ones that require training - supervised and ones that do not - unsupervised. In this study, we are interested in settings, where for a language under investigation, no training data is available. More specifically, we explore whether pretrained multilingual language models can be employed for zero-shot cross-lingual keyword extraction on low-resource languages with limited or no available labeled training data and whether they outperform state-of-the-art unsupervised keyword extractors. The comparison is conducted on six news article datasets covering two high-resource languages, English and Russian, and four low-resource languages, Croatian, Estonian, Latvian, and Slovenian. We find that the pretrained models fine-tuned on a multilingual corpus covering languages that do not appear in the test set (i.e. in a zero-shot setting), consistently outscore unsupervised models in all six languages.