# Assessing generalization capability of text ranking models in Polish

arXiv: https://arxiv.org/abs/2402.14318 · HTML: https://mlsanity.com/paper/2402.14318/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 26 of its 33 references that are in the map.
- First posted on arXiv 22 Feb 2024.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)

## Builds on (most cited first, 26 of 26)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Document Ranking with a Pretrained Sequence-to-Sequence Model](https://mlsanity.com/paper/2003.06713/) (2020)
- [Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents](https://mlsanity.com/paper/2304.09542/) (2023)
- [Large Language Models for Information Retrieval: A Survey](https://mlsanity.com/paper/2308.07107/) (2023)
- [Fine-Tuning LLaMA for Multi-Stage Text Retrieval](https://mlsanity.com/paper/2310.08319/) (2023)
- [Pretrained Transformers for Text Ranking: BERT and Beyond](https://mlsanity.com/paper/2010.06467/) (2020)
- [A Survey on Retrieval-Augmented Text Generation](https://mlsanity.com/paper/2202.01110/) (2022)
- [Dense Text Retrieval based on Pretrained Language Models: A Survey](https://mlsanity.com/paper/2211.14876/) (2022)
- [Mr. TyDi: A Multi-lingual Benchmark for Dense Retrieval](https://mlsanity.com/paper/2108.08787/) (2021)
- [RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses](https://mlsanity.com/paper/2210.10634/) (2022)
- [From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective](https://mlsanity.com/paper/2205.04733/) (2022)
- [A Deep Look into Neural Ranking Models for Information Retrieval](https://mlsanity.com/paper/1903.06902/) (2019)
- [HerBERT: Efficiently Pretrained Transformer-based Language Model for Polish](https://mlsanity.com/paper/2105.01735/) (2021)
- [Pre-training Polish Transformer-based Language Models at Scale](https://mlsanity.com/paper/2006.04229/) (2020)
- [In Defense of Cross-Encoders for Zero-Shot Retrieval](https://mlsanity.com/paper/2212.06121/) (2022)
- [BEIR-PL: Zero Shot Information Retrieval Benchmark for the Polish Language](https://mlsanity.com/paper/2305.19840/) (2023)
- [PIRB: A Comprehensive Benchmark of Polish Dense and Hybrid Text Retrieval Methods](https://mlsanity.com/paper/2402.13350/) (2024)
- [Evaluation of Transfer Learning for Polish with a Text-to-Text Model](https://mlsanity.com/paper/2205.08808/) (2022)
- [MAUPQA: Massive Automatically-created Polish Question Answering Dataset](https://mlsanity.com/paper/2305.05486/) (2023)
- [Silver Retriever: Advancing Neural Passage Retrieval for Polish Question Answering](https://mlsanity.com/paper/2309.08469/) (2023)

## Built on by (most cited first, 3 of 3)

- [Evaluating LLMs Robustness in Less Resourced Languages with Proxy Models](https://mlsanity.com/paper/2506.07645/) (2025)
- [Developing PUGG for Polish: A Modern Approach to KBQA, MRC, and IR Dataset Construction](https://mlsanity.com/paper/2408.02337/) (2024)
- [Optimizing Retrieval-Augmented Generation of Medical Content for Spaced Repetition Learning](https://mlsanity.com/paper/2503.01859/) (2025)

## Abstract (arXiv)

Retrieval-augmented generation (RAG) is becoming an increasingly popular technique for integrating internal knowledge bases with large language models. In a typical RAG pipeline, three models are used, responsible for the retrieval, reranking, and generation stages. In this article, we focus on the reranking problem for the Polish language, examining the performance of rerankers and comparing their results with available retrieval models. We conduct a comprehensive evaluation of existing models and those trained by us, utilizing a benchmark of 41 diverse information retrieval tasks for the Polish language. The results of our experiments show that most models struggle with out-of-domain generalization. However, a combination of effective optimization method and a large training dataset allows for building rerankers that are both compact in size and capable of generalization. The best of our models establishes a new state-of-the-art for reranking in the Polish language, outperforming existing models with up to 30 times more parameters.