mlsanityData to 14 Oct 2026

Leveraging Estimated Transferability Over Human Intuition for Model Selection in Text Ranking

arXiv 2409.16198 ↗First posted 24 Sep 2024cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Text ranking has witnessed significant advancements, attributed to the utilization of dual-encoder enhanced by Pre-trained Language Models (PLMs). Given the proliferation of available PLMs, selecting the most effective one for a given dataset has become a non-trivial challenge. As a promising alternative to human intuition and brute-force fine-tuning, Transferability Estimation (TE) has emerged as an effective approach to model selection. However, current TE methods are primarily designed for classification tasks, and their estimated transferability may not align well with the objectives of text ranking. To address this challenge, we propose to compute the expected rank as transferability, explicitly reflecting the model's ranking capability. Furthermore, to mitigate anisotropy and incorporate training dynamics, we adaptively scale isotropic sentence embeddings to yield an accurate expected rank score. Our resulting method, Adaptive Ranking Transferability (AiRTran), can effectively capture subtle differences between models. On challenging model selection scenarios across various text ranking datasets, it demonstrates significant improvements over previous classification-oriented TE methods, human intuition, and ChatGPT with minor time consumption.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 51 references that are in the map. First posted on arXiv 24 Sep 2024.

Builds on

33 of 51 references.

References · 33

  1. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  2. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  3. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  4. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  5. 2020Fact or Fiction: Verifying Scientific Claimscited by 554
  6. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 463
  7. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 281
  8. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 214
  9. 2021Whitening Sentence Representations for Better Semantics and Faster Retrievalcited by 202
  10. 2021LogME: Practical Assessment of Pre-trained Models for Transfer Learningcited by 172
  11. 2020MuTual: A Dataset for Multi-Turn Dialogue Reasoningcited by 115
  12. 2022From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effectivecited by 98
21 more
  1. 2021Ensembling Off-the-shelf Models for GAN Trainingcited by 86
  2. 2022An Information-Theoretic Approach to Transferability in Task Transfer Learningcited by 81
  3. 2020Relevance-guided Supervision for OpenQA with ColBERTcited by 71
  4. 2021WhiteningBERT: An Easy Unsupervised Sentence Embedding Approachcited by 69
  5. 2020Ranking Neural Checkpointscited by 51
  6. 2020Scalable Transfer Learning with Expert Modelscited by 49
  7. 2023An Empirical Study of Pre-Trained Model Reuse in the Hugging Face Deep Learning Model Registrycited by 47
  8. 2023Learning to Rank in Generative Retrievalcited by 46
  9. 2022Curriculum Learning for Dense Retrieval Distillationcited by 38
  10. 2021Evaluation of BERT and ALBERT Sentence Embedding Performance on Downstream NLP Taskscited by 32
  11. 2023Model Spider: Learning to Rank Pre-Trained Models Efficientlycited by 32
  12. 2022PACTran: PAC-Bayesian Metrics for Estimating the Transferability of Pretrained Models to Classification Taskscited by 30
  13. 2022How stable are Transferability Metrics evaluations?cited by 30
  14. 2022Not All Models Are Equal: Predicting Model Transferability in a Self-challenging Fisher Spacecited by 30
  15. 2022PROD: Progressive Distillation for Dense Retrievalcited by 21
  16. 2022ConTextual Masked Auto-Encoder for Dense Passage Retrievalcited by 16
  17. 2023Foundation Model is Efficient Multimodal Multitask Model Selectorcited by 16
  18. 2022DPTDR: Deep Prompt Tuning for Dense Passage Retrievalcited by 10
  19. 2023Fast and Accurate Transferability Measurement by Evaluating Intra-class Feature Variancecited by 9
  20. 2023How to Determine the Most Powerful Pre-trained Language Model without Brute Force Fine-tuning? An Empirical Surveycited by 4
  21. 2022Enhancing Dual-Encoders with Question and Answer Cross-Embeddings for Answer Retrievalcited by 2

Built on by

Not cited yet.