# ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search

arXiv: https://arxiv.org/abs/2006.05324 · HTML: https://mlsanity.com/paper/2006.05324/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jun 2020.

## Built on by (most cited first, 30 of 34)

- [Overview of the TREC 2020 deep learning track](https://mlsanity.com/paper/2102.07662/) (2021)
- [PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrieval](https://mlsanity.com/paper/2108.06027/) (2021)
- [Search Arena: Analyzing Search-Augmented LLMs](https://mlsanity.com/paper/2506.05334/) (2025)
- [Releasing Graph Neural Networks with Differential Privacy Guarantees](https://mlsanity.com/paper/2109.08907/) (2021)
- [Simplified Data Wrangling with ir_datasets](https://mlsanity.com/paper/2103.02280/) (2021)
- [BASES: Large-scale Web Search User Simulation with Large Language Model based Agents](https://mlsanity.com/paper/2402.17505/) (2024)
- [TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime](https://mlsanity.com/paper/2104.09399/) (2021)
- [QUILL: Query Intent with Large Language Models using Retrieval Augmentation and Multi-stage Distillation](https://mlsanity.com/paper/2210.15718/) (2022)
- [MultiCoNER: A Large-scale Multilingual dataset for Complex Named Entity Recognition](https://mlsanity.com/paper/2208.14536/) (2022)
- [DAMO-NLP at SemEval-2022 Task 11: A Knowledge-based System for Multilingual Named Entity Recognition](https://mlsanity.com/paper/2203.00545/) (2022)
- [vCache: Verified Semantic Prompt Caching](https://mlsanity.com/paper/2502.03771/) (2025)
- [Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspective](https://mlsanity.com/paper/2407.06992/) (2024)
- [PEEK: A Large Dataset of Learner Engagement with Educational Videos](https://mlsanity.com/paper/2109.03154/) (2021)
- [USTC-NELSLIP at SemEval-2022 Task 11: Gazetteer-Adapted Integration Network for Multilingual Complex Named Entity Recognition](https://mlsanity.com/paper/2203.03216/) (2022)
- [Siamese BERT-based Model for Web Search Relevance Ranking Evaluated on a New Czech Dataset](https://mlsanity.com/paper/2112.01810/) (2021)
- [Auditing Google's AI Overviews and Featured Snippets: A Case Study on Baby Care and Pregnancy](https://mlsanity.com/paper/2511.12920/) (2025)
- [Noise-Robust Dense Retrieval via Contrastive Alignment Post Training](https://mlsanity.com/paper/2304.03401/) (2023)
- [A Toolbox for Modelling Engagement with Educational Videos](https://mlsanity.com/paper/2401.05424/) (2023)
- [VectorLiteRAG: Latency-Aware and Fine-Grained Resource Partitioning for Efficient RAG](https://mlsanity.com/paper/2504.08930/) (2025)
- [Modeling Relevance Ranking under the Pre-training and Fine-tuning Paradigm](https://mlsanity.com/paper/2108.05652/) (2021)
- [USTC-NELSLIP at SemEval-2023 Task 2: Statistical Construction and Dual Adaptation of Gazetteer for Multilingual Complex NER](https://mlsanity.com/paper/2305.02517/) (2023)
- [CWRCzech: 100M Query-Document Czech Click Dataset and Its Application to Web Relevance Ranking](https://mlsanity.com/paper/2405.20994/) (2024)
- [Optimizing Novelty of Top-k Recommendations using Large Language Models and Reinforcement Learning](https://mlsanity.com/paper/2406.14169/) (2024)
- [GEO-Flag: Detecting and Measuring GEO-Optimized Web Content](https://mlsanity.com/paper/2608.16824/) (2026)
- [Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence](https://mlsanity.com/paper/2104.09393/) (2021)
- [Can Population-based Engagement Improve Personalisation? A Novel Dataset and Experiments](https://mlsanity.com/paper/2207.01504/) (2022)
- [A Deep Reinforcement Learning Approach for Interactive Search with Sentence-level Feedback](https://mlsanity.com/paper/2310.03043/) (2023)
- [Coarse-Tuning for Ad-hoc Document Retrieval Using Pre-trained Language Models](https://mlsanity.com/paper/2403.16915/) (2024)
- [De-DSI: Decentralised Differentiable Search Index](https://mlsanity.com/paper/2404.12237/) (2024)
- [On the Necessity of World Knowledge for Mitigating Missing Labels in Extreme Classification](https://mlsanity.com/paper/2408.09585/) (2024)

## Abstract (arXiv)

Users of Web search engines reveal their information needs through queries and clicks, making click logs a useful asset for information retrieval. However, click logs have not been publicly released for academic use, because they can be too revealing of personally or commercially sensitive information. This paper describes a click data release related to the TREC Deep Learning Track document corpus. After aggregation and filtering, including a k-anonymity requirement, we find 1.4 million of the TREC DL URLs have 18 million connections to 10 million distinct queries. Our dataset of these queries and connections to TREC documents is of similar size to proprietary datasets used in previous papers on query mining and ranking. We perform some preliminary experiments using the click data to augment the TREC DL training data, offering by comparison: 28x more queries, with 49x more connections to 4.4x more URLs in the corpus. We present a description of the dataset's generation process, characteristics, use in ranking and suggest other potential uses.