mlsanityData to 7 Oct 2026

RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses

arXiv 2210.10634 ↗First posted 12 Oct 2022cs.IR cs.CLcited by 114

Not on the map. Why?

Cited by 27 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recently, substantial progress has been made in text ranking based on pretrained language models such as BERT. However, there are limited studies on how to leverage more powerful sequence-to-sequence models such as T5. Existing attempts usually formulate text ranking as classification and rely on postprocessing to obtain a ranked list. In this paper, we propose RankT5 and study two T5-based ranking model structures, an encoder-decoder and an encoder-only one, so that they not only can directly output ranking scores for each query-document pair, but also can be fine-tuned with "pairwise" or "listwise" ranking losses to optimize ranking performances. Our experiments show that the proposed models with ranking losses can achieve substantial ranking performance gains on different public text ranking data sets. Moreover, when fine-tuned with listwise ranking losses, the ranking model appears to have better zero-shot ranking performance on out-of-domain data sets compared to the model fine-tuned with classification losses.

Key facts. Cited by 114 papers in the map (data through 7 Oct 2026). Builds on 35 of its 58 references that are in the map. First posted on arXiv 12 Oct 2022.

Builds on

35 of 58 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  7. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  8. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  9. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  10. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  11. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  12. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
23 more
  1. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  2. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  3. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
  4. 2020Pretrained Transformers for Text Ranking: BERT and Beyondcited by 266
  5. 2022Transformer Memory as a Differentiable Search Indexcited by 242
  6. 2019Multi-Stage Document Ranking with BERTcited by 200
  7. 2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 175
  8. 2022Improving Passage Retrieval with Zero-Shot Question Generationcited by 164
  9. 2013Introducing LETOR 4.0 Datasetscited by 120
  10. 2022RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encodercited by 118
  11. 2022SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrievalcited by 90
  12. 2021The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Modelscited by 81
  13. 2021Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipelinecited by 65
  14. 2022PolyLoss: A Polynomial Expansion Perspective of Classification Loss Functionscited by 62
  15. 2020Efficient Document Re-Ranking for Transformers by Precomputing Term Representationscited by 44
  16. 2022ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrievalcited by 41
  17. 2020Learning-to-Rank with BERT in TF-Rankingcited by 36
  18. 2022Adaptive Re-Ranking with a Corpus Graphcited by 23
  19. 2020Beyond [CLS] through Ranking by Generationcited by 20
  20. 2022HLATR: Enhance Multi-stage Text Retrieval with Hybrid List Aware Transformer Rerankingcited by 19
  21. 2022No Parameter Left Behind: How Distillation and Model Size Affect Zero-Shot Retrievalcited by 17
  22. 2021Text-to-Text Multi-view Learning for Passage Re-rankingcited by 14
  23. 2022ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inferencecited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 18; 2024: 37; 2025: 33; 2026: 23.

114 papers.

Sort:

By area:

  1. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  2. 2023Large Language Models are Effective Text Rankers with Pairwise Ranking Promptingcited by 302
  3. 2023Zero-Shot Listwise Document Reranking with a Large Language Modelcited by 81
  4. 2024A Survey on Retrieval-Augmented Text Generation for Large Language Modelscited by 77
  5. 2023InPars-v2: Large Language Models as Efficient Dataset Generators for Information Retrievalcited by 73
  6. 2023RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Modelscited by 69
  7. 2022Don't Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environmentscited by 57
  8. 2024Don't Forget to Connect! Improving RAG with Graph-based Rerankingcited by 49
  9. 2023How Does Generative Retrieval Scale to Millions of Passages?cited by 47
  10. 2024TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategycited by 47
  11. 2024CoRNStack: High-Quality Contrastive Data for Better Code Retrieval and Rerankingcited by 37
  12. 2024JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Rerankingcited by 34
38 more
  1. 2023Found in the Middle: Permutation Self-Consistency Improves Listwise Ranking in Large Language Modelscited by 30
  2. 2024APEER: Automatic Prompt Engineering Enhances Large Language Model Rerankingcited by 28
  3. 2024Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generationcited by 27
  4. 2023MILL: Mutual Verification with Large Language Models for Zero-Shot Query Expansioncited by 22
  5. 2024Leveraging Passage Embeddings for Efficient Listwise Reranking with Large Language Modelscited by 20
  6. 2024Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decodingcited by 19
  7. 2024FunnelRAG: A Coarse-to-Fine Progressive Retrieval Paradigm for RAGcited by 18
  8. 2024RichRAG: Crafting Rich Responses for Multi-faceted Queries in Retrieval-Augmented Generationcited by 16
  9. 2024RankMamba: Benchmarking Mamba's Document Ranking Performance in the Era of Transformerscited by 15
  10. 2025Rankers, Judges, and Assistants: Towards Understanding the Interplay of LLMs in Information Retrieval Evaluationcited by 15
  11. 2023Rank-without-GPT: Building GPT-Independent Listwise Rerankers on Open-Source Large Language Modelscited by 14
  12. 2025The RAG Paradox: A Black-Box Attack Exploiting Unintentional Vulnerabilities in Retrieval-Augmented Generation Systemscited by 14
  13. 2025Rankify: A Comprehensive Python Toolkit for Retrieval, Re-Ranking, and Retrieval-Augmented Generationcited by 13
  14. 2025Shifting from Ranking to Set Selection for Retrieval Augmented Generationcited by 13
  15. 2023Investigating semantic subspaces of Transformer sentence embeddings through linear structural probingcited by 12
  16. 2025Optimizing Generative Ranking Relevance via Reinforcement Learning in Xiaohongshu Searchcited by 11
  17. 2024State Space Models are Strong Text Rerankerscited by 10
  18. 2025Adapting Decoder-Based Language Models for Diverse Encoder Downstream Taskscited by 10
  19. 2025CoRanking: Collaborative Ranking with Small and Large Ranking Agentscited by 10
  20. 2023Exploring the Viability of Synthetic Query Generation for Relevance Predictioncited by 9
  21. 2023It's All Relative! -- A Synthetic Query Generation Approach for Improving Zero-Shot Relevance Predictioncited by 9
  22. 2024Improving Pinterest Search Relevance Using Large Language Modelscited by 8
  23. 2024Drowning in Documents: Consequences of Scaling Reranker Inferencecited by 8
  24. 2025AcuRank: Uncertainty-Aware Adaptive Computation for Listwise Rerankingcited by 8
  25. 2024MCRanker: Generating Diverse Criteria On-the-Fly to Improve Point-wise LLM Rankerscited by 7
  26. 2024Self-Calibrated Listwise Reranking with Large Language Modelscited by 7
  27. 2024Advancing Single and Multi-task Text Classification through Large Language Model Fine-tuningcited by 7
  28. 2025LLM Alignment as Retriever Optimization: An Information Retrieval Perspectivecited by 7
  29. 2022HYRR: Hybrid Infused Reranking for Passage Retrievalcited by 6
  30. 2024Multimodal Reranking for Knowledge-Intensive Visual Question Answeringcited by 6
  31. 2024RRADistill: Distilling LLMs' Passage Ranking Ability for Long-Tail Queries Document Re-Ranking on a Search Enginecited by 6
  32. 2025Precise Zero-Shot Pointwise Ranking with LLMs through Post-Aggregated Global Context Informationcited by 6
  33. 2025Distillation versus Contrastive Learning: How to Train Your Rerankerscited by 6
  34. 2024DREQ: Document Re-Ranking Using Entity-based Query Understandingcited by 5
  35. 2024DSLR: Document Refinement with Sentence-Level Re-ranking and Reconstruction to Enhance Retrieval-Augmented Generationcited by 5
  36. 2025EqualizeIR: Mitigating Linguistic Biases in Retrieval Modelscited by 5
  37. 2025CoRank: LLM-Based Compact Reranking with Document Features for Scientific Retrievalcited by 5
  38. 2025Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencingcited by 5

50 most cited of 114 All 114 (JSON)