mlsanityData to 13 Oct 2026

Overview of the TREC 2020 deep learning track

arXiv 2102.07662 ↗First posted 15 Feb 2021cs.IR cs.AI cs.CLcited by 195

Not on the map. Why?

Abstract (arXiv)

This is the second year of the TREC Deep Learning Track, with the goal of studying ad hoc ranking in the large training data regime. We again have a document retrieval task and a passage retrieval task, each with hundreds of thousands of human-labeled training queries. We evaluate using single-shot TREC-style evaluation, to give us a picture of which ranking methods work best when large data is available, with much more comprehensive relevance labeling on the small number of test queries. This year we have further evidence that rankers with BERT-style pretraining outperform other rankers in the large data regime.

Key facts. Cited by 195 papers in the map (data through 13 Oct 2026). Builds on 12 of its 22 references that are in the map. First posted on arXiv 15 Feb 2021.

Builds on

12 of 22 references.

References · 12

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  2. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,810
  3. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,600
  4. 2016Bag of Tricks for Efficient Text Classificationcited by 872
  5. 2019Document Expansion by Query Predictioncited by 226
  6. 2016Learning to Match Using Local and Distributed Representations of Text for Web Searchcited by 68
  7. 2020ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Searchcited by 34
  8. 2019Critically Examining the "Neural Hype": Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Modelscited by 24
  9. 2016Off the Beaten Path: Let's Replace Term-Based Retrieval with k-NN Searchcited by 14
  10. 2019An Updated Duet Model for Passage Re-rankingcited by 12
  11. 2018Optimizing Query Evaluations using Reinforcement Learning for Web Searchcited by 9
  12. 2019Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networkscited by 4

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 16; 2022: 25; 2023: 34; 2024: 34; 2025: 45; 2026: 41.

195 papers.

Sort:

By area:

  1. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 510
  2. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 358
  3. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  4. 2023Query2doc: Query Expansion with Large Language Modelscited by 148
  5. 2021Mr. TyDi: A Multi-lingual Benchmark for Dense Retrievalcited by 119
  6. 2023How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrievalcited by 117
  7. 2021mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Datasetcited by 116
  8. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  9. 2024Searching for Best Practices in Retrieval-Augmented Generationcited by 83
  10. 2023Zero-Shot Listwise Document Reranking with a Large Language Modelcited by 82
  11. 2023RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Modelscited by 70
  12. 2022InPars: Data Augmentation for Information Retrieval using Large Language Modelscited by 60
38 more
  1. 2025Rank-R1: Enhancing Reasoning in LLM-based Document Rerankers via Reinforcement Learningcited by 59
  2. 2023A Setwise Approach for Effective and Highly Efficient Zero-shot Ranking with Large Language Modelscited by 55
  3. 2023How Does Generative Retrieval Scale to Millions of Passages?cited by 47
  4. 2024TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategycited by 47
  5. 2021Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedbackcited by 45
  6. 2022Curriculum Learning for Dense Retrieval Distillationcited by 38
  7. 2024INQUIRE: A Natural World Text-to-Image Retrieval Benchmarkcited by 33
  8. 2023Found in the Middle: Permutation Self-Consistency Improves Listwise Ranking in Large Language Modelscited by 31
  9. 2021Challenges in Generalization in Open Domain Question Answeringcited by 30
  10. 2021MS MARCO: Benchmarking Ranking Models in the Large-Data Regimecited by 28
  11. 2025Rank-K: Test-Time Reasoning for Listwise Rerankingcited by 28
  12. 2021Simplified Data Wrangling with ir_datasetscited by 27
  13. 2022An Efficiency Study for SPLADE Modelscited by 26
  14. 2023Large Language Models are Strong Zero-Shot Retrievercited by 25
  15. 2023When do Generative Query and Document Expansions Fail? A Comprehensive Study Across Methods, Retrievers, and Datasetscited by 25
  16. 2021TREC Deep Learning Track: Reusable Test Collections in the Large Data Regimecited by 23
  17. 2023MILL: Mutual Verification with Large Language Models for Zero-Shot Query Expansioncited by 22
  18. 2025Overview of the TREC 2022 deep learning trackcited by 22
  19. 2022Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reductioncited by 21
  20. 2022In Defense of Cross-Encoders for Zero-Shot Retrievalcited by 21
  21. 2024Ranked List Truncation for Large Language Model-based Re-Rankingcited by 20
  22. 2021Boosting Search Engines with Interactive Agentscited by 19
  23. 2022CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrievalcited by 19
  24. 2023Instruction Distillation Makes Large Language Models Efficient Zero-shot Rankerscited by 19
  25. 2024Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decodingcited by 19
  26. 2024Query Performance Prediction using Relevance Judgments Generated by Large Language Modelscited by 18
  27. 2025Overview of the TREC 2021 deep learning trackcited by 18
  28. 2024Cocktail: A Comprehensive Information Retrieval Benchmark with LLM-Generated Documents Integrationcited by 17
  29. 2024DemoRank: Selecting Effective Demonstrations for Large Language Models in Ranking Taskcited by 17
  30. 2024Corpus-Steered Query Expansion with Large Language Modelscited by 16
  31. 2022LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrievalcited by 15
  32. 2023Constructing Tree-based Index for Efficient and Effective Dense Retrievalcited by 15
  33. 2024RankMamba: Benchmarking Mamba's Document Ranking Performance in the Era of Transformerscited by 15
  34. 2024Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspectivecited by 15
  35. 2025O1 Embedder: Let Retrievers Think Before Actioncited by 15
  36. 2025Rankers, Judges, and Assistants: Towards Understanding the Interplay of LLMs in Information Retrieval Evaluationcited by 15
  37. 2025A Human-AI Comparative Analysis of Prompt Sensitivity in LLM-Based Relevance Judgmentcited by 15
  38. 2023Rank-without-GPT: Building GPT-Independent Listwise Rerankers on Open-Source Large Language Modelscited by 14

50 most cited of 195 All 195 (JSON)