# Exploring Fine-tuned Generative Models for Keyphrase Selection: A Case Study for Russian

arXiv: https://arxiv.org/abs/2409.10640 · HTML: https://mlsanity.com/paper/2409.10640/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 51 references that are in the map.
- First posted on arXiv 16 Sep 2024.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Multilingual Translation with Extensible Multilingual Pretraining and Finetuning](https://mlsanity.com/paper/2008.00401/) (2020)
- [Evaluation of Text Generation: A Survey](https://mlsanity.com/paper/2006.14799/) (2020)
- [Adaptation of Deep Bidirectional Multilingual Transformers for Russian Language](https://mlsanity.com/paper/1905.07213/) (2019)
- [Deep Keyphrase Generation](https://mlsanity.com/paper/1704.06879/) (2017)
- [Keyphrase Generation with Correlation Constraints](https://mlsanity.com/paper/1808.07185/) (2018)
- [Neural Keyphrase Generation via Reinforcement Learning with Adaptive Rewards](https://mlsanity.com/paper/1906.04106/) (2019)
- [Learning Rich Representation of Keyphrases from Text](https://mlsanity.com/paper/2112.08547/) (2021)
- [A Review of Keyphrase Extraction](https://mlsanity.com/paper/1905.05044/) (2019)
- [A Family of Pretrained Transformer Language Models for Russian](https://mlsanity.com/paper/2309.10931/) (2023)
- [Morphological Analyzer and Generator for Russian and Ukrainian Languages](https://mlsanity.com/paper/1503.07283/) (2015)
- [Is ChatGPT A Good Keyphrase Generator? A Preliminary Study](https://mlsanity.com/paper/2303.13001/) (2023)
- [Pre-trained Language Models for Keyphrase Generation: A Thorough Empirical Study](https://mlsanity.com/paper/2212.10233/) (2022)
- [ChatGPT vs State-of-the-Art Models: A Benchmarking Study in Keyphrase Generation Task](https://mlsanity.com/paper/2304.14177/) (2023)
- [Keyphrase Generation: A Text Summarization Struggle](https://mlsanity.com/paper/1904.00110/) (2019)
- [Keyphrase Extraction with Span-based Feature Representations](https://mlsanity.com/paper/2002.05407/) (2020)
- [Keyphrase Generation: A Multi-Aspect Survey](https://mlsanity.com/paper/1910.05059/) (2019)
- [Hyperbolic Relevance Matching for Neural Keyphrase Extraction](https://mlsanity.com/paper/2205.02047/) (2022)
- [Extending Neural Keyword Extraction with TF-IDF tagset matching](https://mlsanity.com/paper/2102.00472/) (2021)
- [Enhancing Keyphrase Generation by BART Finetuning with Splitting and Shuffling](https://mlsanity.com/paper/2309.06726/) (2023)

## Built on by (most cited first, 1 of 1)

- [Key Algorithms for Keyphrase Generation: Instruction-Based LLMs for Russian Scientific Keyphrases](https://mlsanity.com/paper/2410.18040/) (2024)

## Abstract (arXiv)

Keyphrase selection plays a pivotal role within the domain of scholarly texts, facilitating efficient information retrieval, summarization, and indexing. In this work, we explored how to apply fine-tuned generative transformer-based models to the specific task of keyphrase selection within Russian scientific texts. We experimented with four distinct generative models, such as ruT5, ruGPT, mT5, and mBART, and evaluated their performance in both in-domain and cross-domain settings. The experiments were conducted on the texts of Russian scientific abstracts from four domains: mathematics & computer science, history, medicine, and linguistics. The use of generative models, namely mBART, led to gains in in-domain performance (up to 4.9% in BERTScore, 9.0% in ROUGE-1, and 12.2% in F1-score) over three keyphrase extraction baselines for the Russian language. Although the results for cross-domain usage were significantly lower, they still demonstrated the capability to surpass baseline performances in several cases, underscoring the promising potential for further exploration and refinement in this research field.