# Approaches to Semantic Textual Similarity in Slovak Language: From Algorithms to Transformers

arXiv: https://arxiv.org/abs/2602.04659 · HTML: https://mlsanity.com/paper/2602.04659/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 60 references that are in the map.
- First posted on arXiv 4 Feb 2026.

## Builds on (most cited first, 9 of 9)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017)
- [Learning Word Vectors for 157 Languages](https://mlsanity.com/paper/1802.06893/) (2018)
- [A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages](https://mlsanity.com/paper/2006.06202/) (2020)
- [WikiBERT models: deep transfer learning for many languages](https://mlsanity.com/paper/2006.01538/) (2020)
- [Sentence Similarity Based on Contexts](https://mlsanity.com/paper/2105.07623/) (2021)
- [SlovakBERT: Slovak Masked Language Model](https://mlsanity.com/paper/2109.15254/) (2021)
- [Improving word mover's distance by leveraging self-attention matrix](https://mlsanity.com/paper/2211.06229/) (2022)

## Abstract (arXiv)

Semantic textual similarity (STS) plays a crucial role in many natural language processing tasks. While extensively studied in high-resource languages, STS remains challenging for under-resourced languages such as Slovak. This paper presents a comparative evaluation of sentence-level STS methods applied to Slovak, including traditional algorithms, supervised machine learning models, and third-party deep learning tools. We trained several machine learning models using outputs from traditional algorithms as features, with feature selection and hyperparameter tuning jointly guided by artificial bee colony optimization. Finally, we evaluated several third-party tools, including fine-tuned model by CloudNLP, OpenAI's embedding models, GPT-4 model, and pretrained SlovakBERT model. Our findings highlight the trade-offs between different approaches.