# Learning-to-Rank with BERT in TF-Ranking

arXiv: https://arxiv.org/abs/2004.08476 · HTML: https://mlsanity.com/paper/2004.08476/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2020.

## Built on by (most cited first, 30 of 36)

- [RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking](https://mlsanity.com/paper/2110.07367/) (2021)
- [Explaining Answers with Entailment Trees](https://mlsanity.com/paper/2104.08661/) (2021)
- [Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction](https://mlsanity.com/paper/2305.06474/) (2023)
- [RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses](https://mlsanity.com/paper/2210.10634/) (2022)
- [Adversarial Retriever-Ranker for dense text retrieval](https://mlsanity.com/paper/2110.03611/) (2021)
- [The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Models](https://mlsanity.com/paper/2101.05667/) (2021)
- [Entailment Tree Explanations via Iterative Retrieval-Generation Reasoner](https://mlsanity.com/paper/2205.09224/) (2022)
- [Argument Undermining: Counter-Argument Generation by Attacking Weak Premises](https://mlsanity.com/paper/2105.11752/) (2021)
- [QUILL: Query Intent with Large Language Models using Retrieval Augmentation and Multi-stage Distillation](https://mlsanity.com/paper/2210.15718/) (2022)
- [A Neural Pairwise Ranking Model for Readability Assessment](https://mlsanity.com/paper/2203.07450/) (2022)
- [ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inference](https://mlsanity.com/paper/2204.11458/) (2022)
- [Fairness Through Regularization for Learning to Rank](https://mlsanity.com/paper/2102.05996/) (2021)
- [MCRanker: Generating Diverse Criteria On-the-Fly to Improve Point-wise LLM Rankers](https://mlsanity.com/paper/2404.11960/) (2024)
- [Self-Calibrated Listwise Reranking with Large Language Models](https://mlsanity.com/paper/2411.04602/) (2024)
- [On the Challenges of Evaluating Compositional Explanations in Multi-Hop Inference: Relevance, Completeness, and Expert Ratings](https://mlsanity.com/paper/2109.03334/) (2021)
- [SPBERTQA: A Two-Stage Question Answering System Based on Sentence Transformers for Medical Texts](https://mlsanity.com/paper/2206.09600/) (2022)
- [Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing](https://mlsanity.com/paper/2505.17552/) (2025)
- [Learning List-Level Domain-Invariant Representations for Ranking](https://mlsanity.com/paper/2212.10764/) (2022)
- ["It doesn't look good for a date": Transforming Critiques into Preferences for Conversational Recommendation Systems](https://mlsanity.com/paper/2109.07576/) (2021)
- [Explicit and Implicit Semantic Ranking Framework](https://mlsanity.com/paper/2304.04918/) (2023)
- [Entity Insertion in Multilingual Linked Corpora: The Case of Wikipedia](https://mlsanity.com/paper/2410.04254/) (2024)
- [Open-Domain Conversational Search Assistant with Transformers](https://mlsanity.com/paper/2101.08197/) (2021)
- [Comparing Neighbors Together Makes it Easy: Jointly Comparing Multiple Candidates for Efficient and Effective Retrieval](https://mlsanity.com/paper/2405.12801/) (2024)
- [Enhancing Retrieval for ESGLLM via ESG-CID -- A Disclosure Content Index Finetuning Dataset for Mapping GRI and ESRS](https://mlsanity.com/paper/2503.10674/) (2025)
- [Power-up! What Can Generative Models Do for Human Computation Workflows?](https://mlsanity.com/paper/2307.02243/) (2023)
- [Pretrained deep models outperform GBDTs in Learning-To-Rank under label scarcity](https://mlsanity.com/paper/2308.00177/) (2023)
- [DynaSemble: Dynamic Ensembling of Textual and Structure-Based Models for Knowledge Graph Completion](https://mlsanity.com/paper/2311.03780/) (2023)
- [Baby Bear: Seeking a Just Right Rating Scale for Scalar Annotations](https://mlsanity.com/paper/2408.09765/) (2024)
- [Know When to Fuse: Investigating Non-English Hybrid Retrieval in the Legal Domain](https://mlsanity.com/paper/2409.01357/) (2024)
- [Knowledge-driven Answer Generation for Conversational Search](https://mlsanity.com/paper/2104.06892/) (2021)

## Abstract (arXiv)

This paper describes a machine learning algorithm for document (re)ranking, in which queries and documents are firstly encoded using BERT [1], and on top of that a learning-to-rank (LTR) model constructed with TF-Ranking (TFR) [2] is applied to further optimize the ranking performance. This approach is proved to be effective in a public MS MARCO benchmark [3]. Our first two submissions achieve the best performance for the passage re-ranking task [4], and the second best performance for the passage full-ranking task as of April 10, 2020 [5]. To leverage the lately development of pre-trained language models, we recently integrate RoBERTa [6] and ELECTRA [7]. Our latest submissions improve our previously state-of-the-art re-ranking performance by 4.3% [8], and achieve the third best performance for the full-ranking task [9] as of June 8, 2020. Both of them demonstrate the effectiveness of combining ranking losses with BERT representations for document ranking.