Introducing LETOR 4.0 Datasets
Not on the map. Why?
Abstract (arXiv)
LETOR is a package of benchmark data sets for research on LEarning TO Rank, which contains standard features, relevance judgments, data partitioning, evaluation tools, and several baselines. Version 1.0 was released in April 2007. Version 2.0 was released in Dec. 2007. Version 3.0 was released in Dec. 2008. This version, 4.0, was released in July 2009. Very different from previous versions (V3.0 is an update based on V2.0 and V2.0 is an update based on V1.0), LETOR4.0 is a totally new release. It uses the Gov2 web page collection (~25M pages) and two query sets from Million Query track of TREC 2007 and TREC 2008. We call the two query sets MQ2007 and MQ2008 for short. There are about 1700 queries in MQ2007 with labeled documents and about 800 queries in MQ2008 with labeled documents. If you have any questions or suggestions about the datasets, please kindly email us ([email protected]). Our goal is to make the dataset reliable and useful for the community.
Key facts. Cited by 120 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Jun 2013.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 14; 2022: 32; 2023: 16; 2024: 29; 2025: 17; 2026: 12.
120 papers.
Sort:
By area:
- 2021Revisiting Deep Learning Models for Tabular Datacited by 725
- 2021Tabular Data: Deep Learning is Not All You Needcited by 530
- 2022On Embeddings for Numerical Features in Tabular Deep Learningcited by 185
- 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 170
- 2022RankT5: Fine-Tuning T5 for Text Ranking with Ranking Lossescited by 114
- 2024TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensemblingcited by 105
- 2023TabR: Tabular Deep Learning Meets Nearest Neighbors in 2023cited by 73
- 2024When Search Engine Services meet Large Language Models: Visions and Challengescited by 59
- 2022Neural Basis Models for Interpretabilitycited by 56
- 2022OpenFE: Automated Feature Generation with Expert-level Performancecited by 52
- 2022Revisiting Pretraining Objectives for Tabular Deep Learningcited by 43
- 2024SwitchTab: Switched Autoencoders Are Effective Tabular Learnerscited by 37
38 more
- 2021B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrievalcited by 25
- 2022Conformal Off-Policy Prediction in Contextual Banditscited by 25
- 2024Data Augmentation of Contrastive Learning is Estimating Positive-incentive Noisecited by 25
- 2023A Performance-Driven Benchmark for Feature Selection in Tabular Deep Learningcited by 23
- 2022Large-scale Stochastic Optimization of NDCG Surrogates for Deep Learning with Provable Convergencecited by 21
- 2022Doubly-Robust Estimation for Correcting Position-Bias in Click Feedback for Unbiased Learning to Rankcited by 17
- 2022Toward Understanding Privileged Features Distillation in Learning-to-Rankcited by 16
- 2022Distributed Differential Privacy in Multi-Armed Banditscited by 14
- 2023On Differentially Private Federated Linear Contextual Banditscited by 14
- 2021Control Variates for Slate Off-Policy Evaluationcited by 13
- 2022Predictive Inference with Weak Supervisioncited by 13
- 2022Learning-to-Rank at the Speed of Sampling: Plackett-Luce Gradient Estimation With Minimal Computational Complexitycited by 13
- 2024Offline Model-Based Optimization by Learning to Rankcited by 13
- 2024Binning as a Pretext Task: Improving Self-Supervised Learning in Tabular Domainscited by 12
- 2022Introducing the Expohedron for Efficient Pareto-optimal Fairness-Utility Amortizations in Repeated Rankingscited by 11
- 2022Towards Feature Selection for Ranking and Classification Exploiting Quantum Annealerscited by 11
- 2022PTDE: Personalized Training with Distilled Execution for Multi-Agent Reinforcement Learningcited by 11
- 2023Safe Deployment for Counterfactual Learning to Rank with Exposure-Based Risk Minimizationcited by 11
- 2022Uncertainty Quantification for Fairness in Two-Stage Recommender Systemscited by 10
- 2022Multi-Label Learning to Rank through Multi-Objective Optimizationcited by 10
- 2023Optimization of Rank Losses for Image Retrievalcited by 10
- 2024Unbiased Learning to Rank Meets Reality: Lessons from Baidu's Large-Scale Search Datasetcited by 10
- 2022OCD: Learning to Overfit with Conditional Diffusion Modelscited by 9
- 2022Towards Disentangling Relevance and Bias in Unbiased Learning to Rankcited by 9
- 2023RankFormer: Listwise Learning-to-Rank Using Listwide Labelscited by 9
- 2024Reliable Confidence Intervals for Information Retrieval Evaluation Using Generative A.Icited by 9
- 2025Breaking the Top-K Barrier: Advancing Top-K Ranking Metrics Optimization in Recommender Systemscited by 9
- 2022Hierarchical Average Precision Training for Pertinent Image Retrievalcited by 8
- 2024Arithmetic Feature Interaction Is Necessary for Deep Tabular Learningcited by 8
- 2024Asking Multimodal Clarifying Questions in Mixed-Initiative Conversational Searchcited by 8
- 2021Individually Fair Rankingcited by 7
- 2022Interpretability with full complexity by constraining feature informationcited by 7
- 2024Practical and Robust Safety Guarantees for Advanced Counterfactual Learning to Rankcited by 7
- 2021Robust Generalization and Safe Query-Specialization in Counterfactual Learning to Rankcited by 6
- 2021PairRank: Online Pairwise Learning to Rank by Divide-and-Conquercited by 6
- 2023Adaptive Neural Ranking Framework: Toward Maximized Business Goal for Cascade Ranking Systemscited by 6
- 2024Two-stage Risk Control with Application to Ranked Retrievalcited by 6
- 2022Learning Distributionally Robust Models at Scale via Composite Optimizationcited by 5
50 most cited of 120 All 120 (JSON)
- 2026Efficient Offline Learning of Ranking Policies via Top-k Policy Decomposition29 Sep 2026
- 2026Fast A/B/n Testing: Exact Multi-Policy Comparison via Tree-Coupled Feedback Sharing13 Aug 2026
- 2026Structure-aware Relative Policy Optimization for Ranking28 Jul 2026
- 2026Exposure-Based Reinforcement Learning to Rank21 Jul 2026
- 2026Optimal and Efficient Contextual Combinatorial Semi-bandits with General Function Approximation15 Jul 2026
- 2026Representation Curriculum: Stagewise Training for Robust Ranking and Allocation3 Jun 2026
- 2026Metric-agnostic Learning-to-Rank via Boosting and Rank Approximation16 Apr 2026
- 2026DenoiseRank: Learning to Rank by Diffusion Models17 Feb 2026
- 2026DeepMTL2R: A Library for Deep Multi-task Learning to Rank16 Feb 2026
- 2026From Noise to Order: Learning to Rank via Denoising Diffusion12 Feb 2026
- 2026Improved Algorithms for Nash Welfare in Linear Bandits30 Jan 2026
- 2026A Causal Information-Flow Framework for Unbiased Learning-to-Rank9 Jan 2026
38 more
- 2025DFORD: Directional Feedback based Online Ordinal Regression Learning22 Dec 2025
- 2025Hedonic Neurons: A Mechanistic Mapping of Latent Coalitions in Transformer MLPs28 Sep 2025
- 2025A Large-Scale Web Search Dataset for Federated Online Learning to Rank17 Aug 2025
- 2025Breaking the Top-K Barrier: Advancing Top-K Ranking Metrics Optimization in Recommender Systems4 Aug 2025
- 2025Industry Insights from Comparing Deep Learning and GBDT Models for E-Commerce Learning-to-Rank28 Jul 2025
- 2025Learning to Rank with Variable Result Presentation Lengths29 Jun 2025
- 2025Unidentified and Confounded? Understanding Two-Tower Models for Unbiased Learning to Rank25 Jun 2025
- 2025A Control Function Framework for Mitigating Position Bias in Learning to Rank Systems8 Jun 2025
- 2025Random at First, Fast at Last: NTK-Guided Fourier Pre-Processing for Tabular DL3 Jun 2025
- 2025Bounded-Abstention Pairwise Learning to Rank29 May 2025
- 2025Unlearning for Federated Online Learning to Rank: A Reproducibility Study19 May 2025
- 2025Random Client Selection on Contrastive Federated Learning for Tabular Data16 May 2025
- 2025Bipartite Ranking From Multiple Labels: On Loss Versus Label Aggregation15 Apr 2025
- 2025Continual Contrastive Learning on Tabular Data with Out of Distribution19 Mar 2025
- 2025Tensor Product Neural Networks for Functional ANOVA Model21 Feb 2025
- 2025CSP: A Simulator For Multi-Agent Ranking Competitions16 Feb 2025
- 2025Representation Learning on Out of Distribution in Tabular Data14 Feb 2025
- 2024Retrieval-Augmented Generation with Graphs (GraphRAG)31 Dec 2024
- 2024TSPRank: Bridging Pairwise and Listwise Methods with a Bilinear Travelling Salesman Model18 Nov 2024
- 2024TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensembling31 Oct 2024
- 2024Probing Ranking LLMs: A Mechanistic Analysis for Information Retrieval24 Oct 2024
- 2024Offline Model-Based Optimization by Learning to Rank15 Oct 2024
- 2024COS-DPO: Conditioned One-Shot Multi-Objective Fine-Tuning Framework10 Oct 2024
- 2024Scale-Invariant Learning-to-Rank2 Oct 2024
- 2024Generative Pre-trained Ranking Model with Over-parameterization at Web-Scale (Extended Abstract)25 Sep 2024
- 2024Pre-trained Graphformer-based Ranking at Web-scale Search (Extended Abstract)25 Sep 2024
- 2024Proximal Ranking Policy Optimization for Practical Safety in Counterfactual Learning to Rank15 Sep 2024
- 2024Biased Dueling Bandits with Stochastic Delayed Feedback26 Aug 2024
- 2024Data Augmentation of Contrastive Learning is Estimating Positive-incentive Noise19 Aug 2024
- 2024Practical and Robust Safety Guarantees for Advanced Counterfactual Learning to Rank29 Jul 2024
- 2024Multimodal Label Relevance Ranking via Reinforcement Learning18 Jul 2024
- 2024Reliable Confidence Intervals for Information Retrieval Evaluation Using Generative A.I2 Jul 2024
- 2024When Search Engine Services meet Large Language Models: Visions and Challenges28 Jun 2024
- 2024KEHRL: Learning Knowledge-Enhanced Language Representations with Hierarchical Reinforcement Learning24 Jun 2024
- 2024GotFunding: A grant recommendation system based on scientific articles21 May 2024
- 2024Binning as a Pretext Task: Improving Self-Supervised Learning in Tabular Domains13 May 2024
- 2024Two-stage Risk Control with Application to Ranked Retrieval27 Apr 2024
- 2024Estimating the Hessian Matrix of Ranking Objectives for Stochastic Learning to Rank with Gradient Boosted Trees18 Apr 2024
50 newest of 120 All 120 (JSON)
- 2022OCD: Learning to Overfit with Conditional Diffusion Modelscited by 9
- 2026From Noise to Order: Learning to Rank via Denoising Diffusioncited by 0
- 2026DenoiseRank: Learning to Rank by Diffusion Modelscited by 0
- 2026DenoiseRank: Learning to Rank by Diffusion Models17 Feb 2026
- 2026From Noise to Order: Learning to Rank via Denoising Diffusion12 Feb 2026
- 2022OCD: Learning to Overfit with Conditional Diffusion Models2 Oct 2022
- 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 170
- 2024Retrieval-Augmented Generation with Graphs (GraphRAG)31 Dec 2024
- 2024How to Forget Clients in Federated Online Learning to Rank?24 Jan 2024
- 2021Revisiting Deep Learning Models for Tabular Datacited by 725
- 2021Tabular Data: Deep Learning is Not All You Needcited by 530
- 2022On Embeddings for Numerical Features in Tabular Deep Learningcited by 185
- 2022RankT5: Fine-Tuning T5 for Text Ranking with Ranking Lossescited by 114
- 2024TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensemblingcited by 105
- 2023TabR: Tabular Deep Learning Meets Nearest Neighbors in 2023cited by 73
- 2024When Search Engine Services meet Large Language Models: Visions and Challengescited by 59
- 2022Neural Basis Models for Interpretabilitycited by 56
- 2022OpenFE: Automated Feature Generation with Expert-level Performancecited by 52
- 2022Revisiting Pretraining Objectives for Tabular Deep Learningcited by 43
- 2024SwitchTab: Switched Autoencoders Are Effective Tabular Learnerscited by 37
- 2022Conformal Off-Policy Prediction in Contextual Banditscited by 25
12 most cited of 110 All 110 (JSON)
- 2026Efficient Offline Learning of Ranking Policies via Top-k Policy Decomposition29 Sep 2026
- 2026Structure-aware Relative Policy Optimization for Ranking28 Jul 2026
- 2026Exposure-Based Reinforcement Learning to Rank21 Jul 2026
- 2026Optimal and Efficient Contextual Combinatorial Semi-bandits with General Function Approximation15 Jul 2026
- 2026Representation Curriculum: Stagewise Training for Robust Ranking and Allocation3 Jun 2026
- 2026Metric-agnostic Learning-to-Rank via Boosting and Rank Approximation16 Apr 2026
- 2026DeepMTL2R: A Library for Deep Multi-task Learning to Rank16 Feb 2026
- 2026Improved Algorithms for Nash Welfare in Linear Bandits30 Jan 2026
- 2026A Causal Information-Flow Framework for Unbiased Learning-to-Rank9 Jan 2026
- 2025DFORD: Directional Feedback based Online Ordinal Regression Learning22 Dec 2025
- 2025Hedonic Neurons: A Mechanistic Mapping of Latent Coalitions in Transformer MLPs28 Sep 2025
- 2025A Large-Scale Web Search Dataset for Federated Online Learning to Rank17 Aug 2025
12 newest of 110 All 110 (JSON)