mlsanityData to 7 Oct 2026

Transformers for Headline Selection for Russian News Clusters

arXiv 2106.10487 ↗First posted 19 Jun 2021cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we explore various multilingual and Russian pre-trained transformer-based models for the Dialogue Evaluation 2021 shared task on headline selection. Our experiments show that the combined approach is superior to individual multilingual and monolingual models. We present an analysis of a number of ways to obtain sentence embeddings and learn a ranking model on top of them. We achieve the result of 87.28% and 86.60% accuracy for the public and private test sets respectively.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 13 references that are in the map. First posted on arXiv 19 Jun 2021.

Builds on

12 of 13 references.

References · 12

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  5. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  6. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  7. 2018Universal Sentence Encodercited by 485
  8. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  9. 2018CatBoost: gradient boosting with categorical features supportcited by 208
  10. 2019Adaptation of Deep Bidirectional Multilingual Transformers for Russian Languagecited by 109
  11. 2020RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmarkcited by 53
  12. 2021Russian News Clustering and Headline Selection Shared Taskcited by 1

Built on by

Not cited yet.