mlsanityData to 14 Oct 2026

A Comparative Study of Text Retrieval Models on DaReCzech

arXiv 2411.12921 ↗First posted 19 Nov 2024cs.IR cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

This article presents a comprehensive evaluation of 7 off-the-shelf document retrieval models: Splade, Plaid, Plaid-X, SimCSE, Contriever, OpenAI ADA and Gemma2 chosen to determine their performance on the Czech retrieval dataset DaReCzech. The primary objective of our experiments is to estimate the quality of modern retrieval approaches in the Czech language. Our analyses include retrieval quality, speed, and memory footprint. Secondly, we analyze whether it is better to use the model directly in Czech text, or to use machine translation into English, followed by retrieval in English. Our experiments identify the most effective option for Czech information retrieval. The findings revealed notable performance differences among the models, with Gemma22 achieving the highest precision and recall, while Contriever performing poorly. Conclusively, SPLADE and PLAID models offered a balance of efficiency and performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 29 references that are in the map. First posted on arXiv 19 Nov 2024.

Builds on

25 of 29 references.

References · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  4. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  5. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  6. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  7. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
  8. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 578
  9. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
  10. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 361
  11. 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 286
  12. 2022PLAID: An Efficient Engine for Late Interaction Retrievalcited by 101
13 more
  1. 2022From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effectivecited by 98
  2. 2024ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabiccited by 88
  3. 2020SparTerm: Learning Term-based Sparse Representation for Fast Text Retrievalcited by 45
  4. 2021GermanQuAD and GermanDPR: Improving Non-English Question Answering and Passage Retrievalcited by 39
  5. 2022Transfer Learning Approaches for Building Cross-Language Dense Retrieval Modelscited by 36
  6. 2024TurkishMMLU: Measuring Massive Multitask Language Understanding in Turkishcited by 32
  7. 2024The Scandinavian Embedding Benchmarks: Comprehensive Assessment of Multilingual and Monolingual Text Embeddingcited by 26
  8. 2024Translate-Distill: Learning Cross-Language Dense Retrieval by Translation and Distillationcited by 22
  9. 2024PL-MTEB: Polish Massive Text Embedding Benchmarkcited by 19
  10. 2020Announcing CzEng 2.0 Parallel Corpus with over 2 Gigawordscited by 17
  11. 2021Siamese BERT-based Model for Web Search Relevance Ranking Evaluated on a New Czech Datasetcited by 7
  12. 2023Some Like It Small: Czech Semantic Embedding Models for Industry Applicationscited by 3
  13. 2024CWRCzech: 100M Query-Document Czech Click Dataset and Its Application to Web Relevance Rankingcited by 2

Built on by

Not cited yet.