# Overview of the TREC 2020 deep learning track

arXiv: https://arxiv.org/abs/2102.07662 · HTML: https://mlsanity.com/paper/2102.07662/

## Key facts

- Cited by 196 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 15 Feb 2021.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Document Expansion by Query Prediction](https://mlsanity.com/paper/1904.08375/) (2019)
- [Learning to Match Using Local and Distributed Representations of Text for Web Search](https://mlsanity.com/paper/1610.08136/) (2016)
- [ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search](https://mlsanity.com/paper/2006.05324/) (2020)
- [Critically Examining the "Neural Hype": Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models](https://mlsanity.com/paper/1904.09171/) (2019)
- [Off the Beaten Path: Let's Replace Term-Based Retrieval with k-NN Search](https://mlsanity.com/paper/1610.10001/) (2016)
- [An Updated Duet Model for Passage Re-ranking](https://mlsanity.com/paper/1903.07666/) (2019)
- [Optimizing Query Evaluations using Reinforcement Learning for Web Search](https://mlsanity.com/paper/1804.04410/) (2018)
- [Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networks](https://mlsanity.com/paper/1907.03693/) (2019)

## Built on by (most cited first, 30 of 196)

- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://mlsanity.com/paper/2212.10496/) (2022)
- [Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents](https://mlsanity.com/paper/2304.09542/) (2023)
- [Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling](https://mlsanity.com/paper/2104.06967/) (2021)
- [Query2doc: Query Expansion with Large Language Models](https://mlsanity.com/paper/2303.07678/) (2023)
- [Mr. TyDi: A Multi-lingual Benchmark for Dense Retrieval](https://mlsanity.com/paper/2108.08787/) (2021)
- [How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrieval](https://mlsanity.com/paper/2302.07452/) (2023)
- [mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Dataset](https://mlsanity.com/paper/2108.13897/) (2021)
- [From Matching to Generation: A Survey on Generative Information Retrieval](https://mlsanity.com/paper/2404.14851/) (2024)
- [Searching for Best Practices in Retrieval-Augmented Generation](https://mlsanity.com/paper/2407.01219/) (2024)
- [Zero-Shot Listwise Document Reranking with a Large Language Model](https://mlsanity.com/paper/2305.02156/) (2023)
- [RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Models](https://mlsanity.com/paper/2309.15088/) (2023)
- [InPars: Data Augmentation for Information Retrieval using Large Language Models](https://mlsanity.com/paper/2202.05144/) (2022)
- [Rank-R1: Enhancing Reasoning in LLM-based Document Rerankers via Reinforcement Learning](https://mlsanity.com/paper/2503.06034/) (2025)
- [A Setwise Approach for Effective and Highly Efficient Zero-shot Ranking with Large Language Models](https://mlsanity.com/paper/2310.09497/) (2023)
- [How Does Generative Retrieval Scale to Millions of Passages?](https://mlsanity.com/paper/2305.11841/) (2023)
- [Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback](https://mlsanity.com/paper/2108.13454/) (2021)
- [TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategy](https://mlsanity.com/paper/2406.11678/) (2024)
- [Curriculum Learning for Dense Retrieval Distillation](https://mlsanity.com/paper/2204.13679/) (2022)
- [INQUIRE: A Natural World Text-to-Image Retrieval Benchmark](https://mlsanity.com/paper/2411.02537/) (2024)
- [Found in the Middle: Permutation Self-Consistency Improves Listwise Ranking in Large Language Models](https://mlsanity.com/paper/2310.07712/) (2023)
- [Challenges in Generalization in Open Domain Question Answering](https://mlsanity.com/paper/2109.01156/) (2021)
- [Rank-K: Test-Time Reasoning for Listwise Reranking](https://mlsanity.com/paper/2505.14432/) (2025)
- [MS MARCO: Benchmarking Ranking Models in the Large-Data Regime](https://mlsanity.com/paper/2105.04021/) (2021)
- [Simplified Data Wrangling with ir_datasets](https://mlsanity.com/paper/2103.02280/) (2021)
- [An Efficiency Study for SPLADE Models](https://mlsanity.com/paper/2207.03834/) (2022)
- [Large Language Models are Strong Zero-Shot Retriever](https://mlsanity.com/paper/2304.14233/) (2023)
- [When do Generative Query and Document Expansions Fail? A Comprehensive Study Across Methods, Retrievers, and Datasets](https://mlsanity.com/paper/2309.08541/) (2023)
- [TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime](https://mlsanity.com/paper/2104.09399/) (2021)
- [MILL: Mutual Verification with Large Language Models for Zero-Shot Query Expansion](https://mlsanity.com/paper/2310.19056/) (2023)
- [Overview of the TREC 2022 deep learning track](https://mlsanity.com/paper/2507.10865/) (2025)

## Abstract (arXiv)

This is the second year of the TREC Deep Learning Track, with the goal of studying ad hoc ranking in the large training data regime. We again have a document retrieval task and a passage retrieval task, each with hundreds of thousands of human-labeled training queries. We evaluate using single-shot TREC-style evaluation, to give us a picture of which ranking methods work best when large data is available, with much more comprehensive relevance labeling on the small number of test queries. This year we have further evidence that rankers with BERT-style pretraining outperform other rankers in the large data regime.