mlsanityData to 7 Oct 2026

Assessing generalization capability of text ranking models in Polish

arXiv 2402.14318 ↗First posted 22 Feb 2024cs.CLcited by 3

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Retrieval-augmented generation (RAG) is becoming an increasingly popular technique for integrating internal knowledge bases with large language models. In a typical RAG pipeline, three models are used, responsible for the retrieval, reranking, and generation stages. In this article, we focus on the reranking problem for the Polish language, examining the performance of rerankers and comparing their results with available retrieval models. We conduct a comprehensive evaluation of existing models and those trained by us, utilizing a benchmark of 41 diverse information retrieval tasks for the Polish language. The results of our experiments show that most models struggle with out-of-domain generalization. However, a combination of effective optimization method and a large training dataset allows for building rerankers that are both compact in size and capable of generalization. The best of our models establishes a new state-of-the-art for reranking in the Polish language, outperforming existing models with up to 30 times more parameters.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 26 of its 33 references that are in the map. First posted on arXiv 22 Feb 2024.

Builds on

26 of 33 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022A Survey on Retrieval-Augmented Text Generation2 Feb

Other references · 25

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  3. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  4. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  5. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  6. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  7. 2019Passage Re-ranking with BERTcited by 629
  8. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
  9. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 356
  10. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  11. 2023Fine-Tuning LLaMA for Multi-Stage Text Retrievalcited by 271
  12. 2020Pretrained Transformers for Text Ranking: BERT and Beyondcited by 266
13 more
  1. 2022Dense Text Retrieval based on Pretrained Language Models: A Surveycited by 136
  2. 2021Mr. TyDi: A Multi-lingual Benchmark for Dense Retrievalcited by 118
  3. 2022RankT5: Fine-Tuning T5 for Text Ranking with Ranking Lossescited by 114
  4. 2022From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effectivecited by 98
  5. 2019A Deep Look into Neural Ranking Models for Information Retrievalcited by 79
  6. 2021HerBERT: Efficiently Pretrained Transformer-based Language Model for Polishcited by 51
  7. 2020Pre-training Polish Transformer-based Language Models at Scalecited by 21
  8. 2022In Defense of Cross-Encoders for Zero-Shot Retrievalcited by 21
  9. 2023BEIR-PL: Zero Shot Information Retrieval Benchmark for the Polish Languagecited by 15
  10. 2024PIRB: A Comprehensive Benchmark of Polish Dense and Hybrid Text Retrieval Methodscited by 14
  11. 2022Evaluation of Transfer Learning for Polish with a Text-to-Text Modelcited by 13
  12. 2023MAUPQA: Massive Automatically-created Polish Question Answering Datasetcited by 11
  13. 2023Silver Retriever: Advancing Neural Passage Retrieval for Polish Question Answeringcited by 5

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.

  1. 2025Evaluating LLMs Robustness in Less Resourced Languages with Proxy Modelscited by 2
  2. 2024Developing PUGG for Polish: A Modern Approach to KBQA, MRC, and IR Dataset Constructioncited by 1
  3. 2025Optimizing Retrieval-Augmented Generation of Medical Content for Spaced Repetition Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond the Reranker: Do RAG Retrieval Enhancements Help Once a Strong Reranker Is Present?
  2. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey
  3. 2025RankLLM: A Python Package for Reranking with LLMs
  4. 2024Towards Optimizing a Retrieval Augmented Generation using Large Language Model on Academic Data
  5. 2024Don't Forget to Connect! Improving RAG with Graph-based Reranking
  6. 2026Rethinking Retrieval-Augmented Generation as a Cooperative Decision-Making Problem
  7. 2025The Evolution of Reranking Models in Information Retrieval: From Heuristic Methods to Large Language Models
  8. 2024Evaluation of Retrieval-Augmented Generation: A Survey