# Generative Pseudo-Labeling for Pre-Ranking with LLMs

arXiv: https://arxiv.org/abs/2602.20995 · HTML: https://mlsanity.com/paper/2602.20995/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 47 references that are in the map.
- First posted on arXiv 24 Feb 2026.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Autoregressive Image Generation using Residual Quantization](https://mlsanity.com/paper/2203.01941/) (2022)
- [Beam Search Strategies for Neural Machine Translation](https://mlsanity.com/paper/1702.01806/) (2017)
- [Causal Intervention for Leveraging Popularity Bias in Recommendation](https://mlsanity.com/paper/2105.06067/) (2021)
- [Multi-Interest Network with Dynamic Routing for Recommendation at Tmall](https://mlsanity.com/paper/1904.08030/) (2019)
- [Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders](https://mlsanity.com/paper/2210.12316/) (2022)
- [LLM-Powered User Simulator for Recommender System](https://mlsanity.com/paper/2412.16984/) (2024)
- [Correcting for Selection Bias in Learning-to-rank Systems](https://mlsanity.com/paper/2001.11358/) (2020)
- [Enhancing Job Recommendation through LLM-based Generative Adversarial Networks](https://mlsanity.com/paper/2307.10747/) (2023)
- [Addressing Confounding Feature Issue for Causal Recommendation](https://mlsanity.com/paper/2205.06532/) (2022)
- [Improved Residual Vector Quantization for High-dimensional Approximate Nearest Neighbor Search](https://mlsanity.com/paper/1509.05195/) (2015)
- [Rec4Ad: A Free Lunch to Mitigate Sample Selection Bias for Ads CTR Prediction in Taobao](https://mlsanity.com/paper/2306.03527/) (2023)
- [Bursting Filter Bubble: Enhancing Serendipity Recommendations with Aligned Large Language Models](https://mlsanity.com/paper/2502.13539/) (2025)
- [MUSE: A Simple Yet Effective Multimodal Search-Based Framework for Lifelong User Interest Modeling](https://mlsanity.com/paper/2512.07216/) (2025)
- [Bounding System-Induced Biases in Recommender Systems with A Randomized Dataset](https://mlsanity.com/paper/2303.11574/) (2023)
- [A Hybrid Cross-Stage Coordination Pre-ranking Model for Online Recommendation Systems](https://mlsanity.com/paper/2502.10284/) (2025)

## Abstract (arXiv)

Pre-ranking is a critical stage in industrial recommendation systems, tasked with efficiently scoring thousands of recalled items for downstream ranking. A key challenge is the train-serving discrepancy: pre-ranking models are trained only on exposed interactions, yet must score all recalled candidates -- including unexposed items -- during online serving. This mismatch not only induces severe sample selection bias but also degrades generalization, especially for long-tail content. Existing debiasing approaches typically rely on heuristics (e.g., negative sampling) or distillation from biased rankers, which either mislabel plausible unexposed items as negatives or propagate exposure bias into pseudo-labels. In this work, we propose Generative Pseudo-Labeling (GPL), a framework that leverages large language models (LLMs) to generate unbiased, content-aware pseudo-labels for unexposed items, explicitly aligning the training distribution with the online serving space. By offline generating user-specific interest anchors and matching them with candidates in a frozen semantic space, GPL provides high-quality supervision without adding online latency. Deployed in a large-scale production system, GPL improves click-through rate by 3.07%, while significantly enhancing recommendation diversity and long-tail item discovery.