# Searching for Scientific Evidence in a Pandemic: An Overview of TREC-COVID

arXiv: https://arxiv.org/abs/2104.09632 · HTML: https://mlsanity.com/paper/2104.09632/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Apr 2021.

## Built on by (most cited first, 19 of 19)

- [Generative Large Language Models in Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2407.02351/) (2024)
- [EXIT: Context-Aware Extractive Compression for Enhancing Retrieval-Augmented Generation](https://mlsanity.com/paper/2412.12559/) (2024)
- [Wikiformer: Pre-training with Structured Information of Wikipedia for Ad-hoc Retrieval](https://mlsanity.com/paper/2312.10661/) (2023)
- [How Can Graph Neural Networks Help Document Retrieval: A Case Study on CORD19 with Concept Map Generation](https://mlsanity.com/paper/2201.04672/) (2022)
- [Hypencoder: Hypernetworks for Information Retrieval](https://mlsanity.com/paper/2502.05364/) (2025)
- [Benchmarking Information Retrieval Models on Complex Retrieval Tasks](https://mlsanity.com/paper/2509.07253/) (2025)
- [100x Cost & Latency Reduction: Performance Analysis of AI Query Approximation using Lightweight Proxy Models](https://mlsanity.com/paper/2603.15970/) (2026)
- [When Embedding Models Meet: Procrustes Bounds and Applications](https://mlsanity.com/paper/2510.13406/) (2025)
- [DIRC-RAG: Accelerating Edge RAG with Robust High-Density and High-Loading-Bandwidth Digital In-ReRAM Computation](https://mlsanity.com/paper/2510.25278/) (2025)
- [A Representation Sharpening Framework for Zero Shot Dense Retrieval](https://mlsanity.com/paper/2511.05684/) (2025)
- [Layer-wise Representation Dynamics: An Empirical Investigation Across Embedders and Base LLMs](https://mlsanity.com/paper/2605.12714/) (2026)
- [Your UnEmbedding Matrix is Secretly a Feature Lens for Text Embeddings](https://mlsanity.com/paper/2606.07502/) (2026)
- [Advancing Chinese biomedical text mining with community challenges](https://mlsanity.com/paper/2403.04261/) (2024)
- [Evaluating Medical LLMs by Levels of Autonomy: A Survey Moving from Benchmarks to Applications](https://mlsanity.com/paper/2510.17764/) (2025)
- [Modular Expert Merging for Biomedical Retrieval](https://mlsanity.com/paper/2602.04731/) (2026)
- [SoftMatcha 2: A Fast and Soft Pattern Matcher for Trillion-Scale Corpora](https://mlsanity.com/paper/2602.10908/) (2026)
- [Hypencoder Revisited: Reproducibility and Analysis of Non-Linear Scoring for First-Stage Retrieval](https://mlsanity.com/paper/2604.27037/) (2026)
- [HAKARI-Bench: A Lightweight Benchmark for Comparing Retrieval Architectures and Efficiency Settings under Unified Conditions](https://mlsanity.com/paper/2606.22778/) (2026)
- [Retrieval, not hallucinations, will be the limiting factor for LLM-based clinical AI tools](https://mlsanity.com/paper/2607.24793/) (2026)

## Abstract (arXiv)

We present an overview of the TREC-COVID Challenge, an information retrieval (IR) shared task to evaluate search on scientific literature related to COVID-19. The goals of TREC-COVID include the construction of a pandemic search test collection and the evaluation of IR methods for COVID-19. The challenge was conducted over five rounds from April to July, 2020, with participation from 92 unique teams and 556 individual submissions. A total of 50 topics (sets of related queries) were used in the evaluation, starting at 30 topics for Round 1 and adding 5 new topics per round to target emerging topics at that state of the still-emerging pandemic. This paper provides a comprehensive overview of the structure and results of TREC-COVID. Specifically, the paper provides details on the background, task structure, topic structure, corpus, participation, pooling, assessment, judgments, results, top-performing systems, lessons learned, and benchmark datasets.