# Overview of the TREC 2019 deep learning track

arXiv: https://arxiv.org/abs/2003.07820 · HTML: https://mlsanity.com/paper/2003.07820/

## Key facts

- Cited by 286 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Mar 2020.

## Built on by (most cited first, 30 of 286)

- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://mlsanity.com/paper/2212.10496/) (2022)
- [BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval](https://mlsanity.com/paper/2407.12883/) (2024)
- [Query2doc: Query Expansion with Large Language Models](https://mlsanity.com/paper/2303.07678/) (2023)
- [SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval](https://mlsanity.com/paper/2109.10086/) (2021)
- [HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly](https://mlsanity.com/paper/2410.02694/) (2024)
- [Mr. TyDi: A Multi-lingual Benchmark for Dense Retrieval](https://mlsanity.com/paper/2108.08787/) (2021)
- [How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrieval](https://mlsanity.com/paper/2302.07452/) (2023)
- [mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Dataset](https://mlsanity.com/paper/2108.13897/) (2021)
- [COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List](https://mlsanity.com/paper/2104.07186/) (2021)
- [On the Theoretical Limitations of Embedding-Based Retrieval](https://mlsanity.com/paper/2508.21038/) (2025)
- [From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective](https://mlsanity.com/paper/2205.04733/) (2022)
- [From Matching to Generation: A Survey on Generative Information Retrieval](https://mlsanity.com/paper/2404.14851/) (2024)
- [Searching for Best Practices in Retrieval-Augmented Generation](https://mlsanity.com/paper/2407.01219/) (2024)
- [Zero-Shot Listwise Document Reranking with a Large Language Model](https://mlsanity.com/paper/2305.02156/) (2023)
- [RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Models](https://mlsanity.com/paper/2309.15088/) (2023)
- [Rank1: Test-Time Compute for Reranking in Information Retrieval](https://mlsanity.com/paper/2502.18418/) (2025)
- [SAILER: Structure-aware Pre-trained Language Model for Legal Case Retrieval](https://mlsanity.com/paper/2304.11370/) (2023)
- [Rank-R1: Enhancing Reasoning in LLM-based Document Rerankers via Reinforcement Learning](https://mlsanity.com/paper/2503.06034/) (2025)
- [A Setwise Approach for Effective and Highly Efficient Zero-shot Ranking with Large Language Models](https://mlsanity.com/paper/2310.09497/) (2023)
- [How Does Generative Retrieval Scale to Millions of Passages?](https://mlsanity.com/paper/2305.11841/) (2023)
- [Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback](https://mlsanity.com/paper/2108.13454/) (2021)
- [TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategy](https://mlsanity.com/paper/2406.11678/) (2024)
- [Attention in Large Language Models Yields Efficient Zero-Shot Re-Rankers](https://mlsanity.com/paper/2410.02642/) (2024)
- [A Statutory Article Retrieval Dataset in French](https://mlsanity.com/paper/2108.11792/) (2021)
- [Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Models](https://mlsanity.com/paper/2409.11136/) (2024)
- [Curriculum Learning for Dense Retrieval Distillation](https://mlsanity.com/paper/2204.13679/) (2022)
- [Towards Robust Ranker for Text Retrieval](https://mlsanity.com/paper/2206.08063/) (2022)
- [Order-Disorder: Imitation Adversarial Attacks for Black-box Neural Ranking Models](https://mlsanity.com/paper/2209.06506/) (2022)
- [Societal Biases in Retrieved Contents: Measurement Framework and Adversarial Mitigation for BERT Rankers](https://mlsanity.com/paper/2104.13640/) (2021)
- [INQUIRE: A Natural World Text-to-Image Retrieval Benchmark](https://mlsanity.com/paper/2411.02537/) (2024)

## Abstract (arXiv)

The Deep Learning Track is a new track for TREC 2019, with the goal of studying ad hoc ranking in a large data regime. It is the first track with large human-labeled training sets, introducing two sets corresponding to two tasks, each with rigorous TREC-style blind evaluation and reusable test sets. The document retrieval task has a corpus of 3.2 million documents with 367 thousand training queries, for which we generate a reusable test set of 43 queries. The passage retrieval task has a corpus of 8.8 million passages with 503 thousand training queries, for which we generate a reusable test set of 43 queries. This year 15 groups submitted a total of 75 runs, using various combinations of deep learning, transfer learning and traditional IR ranking methods. Deep learning runs significantly outperformed traditional IR runs. Possible explanations for this result are that we introduced large training data and we included deep models trained on such data in our judging pools, whereas some past studies did not have such training data or pooling.