# Key Algorithms for Keyphrase Generation: Instruction-Based LLMs for Russian Scientific Keyphrases

arXiv: https://arxiv.org/abs/2410.18040 · HTML: https://mlsanity.com/paper/2410.18040/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 48 references that are in the map.
- First posted on arXiv 23 Oct 2024.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Galactica: A Large Language Model for Science](https://mlsanity.com/paper/2211.09085/) (2022)
- [The Falcon Series of Open Language Models](https://mlsanity.com/paper/2311.16867/) (2023)
- [Multilingual Translation with Extensible Multilingual Pretraining and Finetuning](https://mlsanity.com/paper/2008.00401/) (2020)
- [Why We Need New Evaluation Metrics for NLG](https://mlsanity.com/paper/1707.06875/) (2017)
- [Re-evaluating Evaluation in Text Summarization](https://mlsanity.com/paper/2010.07100/) (2020)
- [Understanding and Mitigating Language Confusion in LLMs](https://mlsanity.com/paper/2406.20052/) (2024)
- [How to Prompt? Opportunities and Challenges of Zero- and Few-Shot Learning for Human-AI Interaction in Creative Applications of Generative Models](https://mlsanity.com/paper/2209.01390/) (2022)
- [MERA: A Comprehensive LLM Evaluation in Russian](https://mlsanity.com/paper/2401.04531/) (2024)
- [Is ChatGPT A Good Keyphrase Generator? A Preliminary Study](https://mlsanity.com/paper/2303.13001/) (2023)
- [ChatGPT vs State-of-the-Art Models: A Benchmarking Study in Keyphrase Generation Task](https://mlsanity.com/paper/2304.14177/) (2023)
- [Vikhr: The Family of Open-Source Instruction-Tuned Large Language Models for Russian](https://mlsanity.com/paper/2405.13929/) (2024)
- [Beyond Metrics: Evaluating LLMs' Effectiveness in Culturally Nuanced, Low-Resource Real-World Scenarios](https://mlsanity.com/paper/2406.00343/) (2024)
- [Comparative Study of Domain Driven Terms Extraction Using Large Language Models](https://mlsanity.com/paper/2404.02330/) (2024)
- [Large Language Models as Zero-Shot Keyphrase Extractors: A Preliminary Empirical Study](https://mlsanity.com/paper/2312.15156/) (2023)
- [Pre-Trained Language Models for Keyphrase Prediction: A Review](https://mlsanity.com/paper/2409.01087/) (2024)
- [Exploring Fine-tuned Generative Models for Keyphrase Selection: A Case Study for Russian](https://mlsanity.com/paper/2409.10640/) (2024)

## Abstract (arXiv)

Keyphrase selection is a challenging task in natural language processing that has a wide range of applications. Adapting existing supervised and unsupervised solutions for the Russian language faces several limitations due to the rich morphology of Russian and the limited number of training datasets available. Recent studies conducted on English texts show that large language models (LLMs) successfully address the task of generating keyphrases. LLMs allow achieving impressive results without task-specific fine-tuning, using text prompts instead. In this work, we access the performance of prompt-based methods for generating keyphrases for Russian scientific abstracts. First, we compare the performance of zero-shot and few-shot prompt-based methods, fine-tuned models, and unsupervised methods. Then we assess strategies for selecting keyphrase examples in a few-shot setting. We present the outcomes of human evaluation of the generated keyphrases and analyze the strengths and weaknesses of the models through expert assessment. Our results suggest that prompt-based methods can outperform common baselines even using simple text prompts.