# Generative Pre-trained Ranking Model with Over-parameterization at Web-Scale (Extended Abstract)

arXiv: https://arxiv.org/abs/2409.16594 · HTML: https://mlsanity.com/paper/2409.16594/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 46 of its 97 references that are in the map.
- First posted on arXiv 25 Sep 2024.

## Builds on (most cited first, 30 of 46)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://mlsanity.com/paper/2008.13535/) (2020)
- [Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation](https://mlsanity.com/paper/2105.14368/) (2021)
- [Dual Aggregation Transformer for Image Super-Resolution](https://mlsanity.com/paper/2308.03364/) (2023)
- [Introducing LETOR 4.0 Datasets](https://mlsanity.com/paper/1306.2597/) (2013)
- [AUC Maximization in the Era of Big Data and AI: A Survey](https://mlsanity.com/paper/2203.15046/) (2022)
- [AgentReview: Exploring Peer Review Dynamics with LLM Agents](https://mlsanity.com/paper/2406.12708/) (2024)
- [Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries](https://mlsanity.com/paper/2310.13132/) (2023)
- [When Search Engine Services meet Large Language Models: Visions and Challenges](https://mlsanity.com/paper/2407.00128/) (2024)
- [Robustifying Safety-Aligned Large Language Models through Clean Data Curation](https://mlsanity.com/paper/2405.19358/) (2024)
- [NeuralNDCG: Direct Optimisation of a Ranking Metric via Differentiable Relaxation of Sorting](https://mlsanity.com/paper/2102.07831/) (2021)
- [Time Series Modeling for Heart Rate Prediction: From ARIMA to Transformers](https://mlsanity.com/paper/2406.12199/) (2024)
- [MM-Soc: Benchmarking Multimodal Large Language Models in Social Media Platforms](https://mlsanity.com/paper/2402.14154/) (2024)
- [CATS: Enhancing Multivariate Time Series Forecasting by Constructing Auxiliary Time Series as Exogenous Variables](https://mlsanity.com/paper/2403.01673/) (2024)
- [Context-Aware Learning to Rank with Self-Attention](https://mlsanity.com/paper/2005.10084/) (2020)
- [ST-MLP: A Cascaded Spatio-Temporal Linear Framework with Channel-Independence Strategy for Traffic Forecasting](https://mlsanity.com/paper/2308.07496/) (2023)
- [Model Extraction Attacks Revisited](https://mlsanity.com/paper/2312.05386/) (2023)
- [MAPO: Boosting Large Language Model Performance with Model-Adaptive Prompt Optimization](https://mlsanity.com/paper/2407.04118/) (2024)
- [Enhance Image-to-Image Generation with LLaVA-generated Prompts](https://mlsanity.com/paper/2406.01956/) (2024)
- [Semantic Communications for Image Recovery and Classification via Deep Joint Source and Channel Coding](https://mlsanity.com/paper/2304.02317/) (2023)
- [Grow-and-Clip: Informative-yet-Concise Evidence Distillation for Answer Explanation](https://mlsanity.com/paper/2201.05088/) (2022)
- [Towards Rationality in Language and Multimodal Agents: A Survey](https://mlsanity.com/paper/2406.00252/) (2024)
- [WildfireGPT: Tailored Large Language Model for Wildfire Analysis](https://mlsanity.com/paper/2402.07877/) (2024)
- [XMeCap: Meme Caption Generation with Sub-Image Adaptability](https://mlsanity.com/paper/2407.17152/) (2024)
- [Rethinking Resource Management in Edge Learning: A Joint Pre-training and Fine-tuning Design Paradigm](https://mlsanity.com/paper/2404.00836/) (2024)
- [Can Pre-trained Language Models Understand Chinese Humor?](https://mlsanity.com/paper/2407.04105/) (2024)
- [Hadamard Adapter: An Extreme Parameter-Efficient Adapter Tuning Method for Pre-trained Language Models](https://mlsanity.com/paper/2407.11033/) (2024)
- [Detecting Misinformation in Multimedia Content through Cross-Modal Entity Consistency: A Dual Learning Approach](https://mlsanity.com/paper/2409.00022/) (2024)

## Abstract (arXiv)

Learning to rank (LTR) is widely employed in web searches to prioritize pertinent webpages from retrieved content based on input queries. However, traditional LTR models encounter two principal obstacles that lead to suboptimal performance: (1) the lack of well-annotated query-webpage pairs with ranking scores covering a diverse range of search query popularities, which hampers their ability to address queries across the popularity spectrum, and (2) inadequately trained models that fail to induce generalized representations for LTR, resulting in overfitting. To address these challenges, we propose a emphulineGenerative ulineSemi-ulineSupervised ulinePre-trained (GS2P) LTR model. We conduct extensive offline experiments on both a publicly available dataset and a real-world dataset collected from a large-scale search engine. Furthermore, we deploy GS2P in a large-scale web search engine with realistic traffic, where we observe significant improvements in the real-world application.