# RRADistill: Distilling LLMs' Passage Ranking Ability for Long-Tail Queries Document Re-Ranking on a Search Engine

arXiv: https://arxiv.org/abs/2410.18097 · HTML: https://mlsanity.com/paper/2410.18097/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge distillation for LLMs, one of 12 directions in Knowledge distillation.
- Builds on 30 of its 48 references that are in the map.
- First posted on arXiv 8 Oct 2024.

## Placed in

- Knowledge distillation / [Knowledge distillation for LLMs](https://mlsanity.com/distill/knowledge-distillation-for-llms/)

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Document Ranking with a Pretrained Sequence-to-Sequence Model](https://mlsanity.com/paper/2003.06713/) (2020)
- [Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting](https://mlsanity.com/paper/2306.17563/) (2023)
- [Overview of the TREC 2019 deep learning track](https://mlsanity.com/paper/2003.07820/) (2020)
- [SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking](https://mlsanity.com/paper/2107.05720/) (2021)
- [Overview of the TREC 2020 deep learning track](https://mlsanity.com/paper/2102.07662/) (2021)
- [RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!](https://mlsanity.com/paper/2312.02724/) (2023)
- [RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses](https://mlsanity.com/paper/2210.10634/) (2022)
- [Beyond Yes and No: Improving Zero-Shot LLM Rankers via Scoring Fine-Grained Relevance Labels](https://mlsanity.com/paper/2310.14122/) (2023)
- [Large Language Models Know Your Contextual Search Intent: A Prompting Framework for Conversational Search](https://mlsanity.com/paper/2303.06573/) (2023)
- [AutoField: Automating Feature Selection in Deep Recommender Systems](https://mlsanity.com/paper/2204.09078/) (2022)
- [DE-RRD: A Knowledge Distillation Framework for Recommender System](https://mlsanity.com/paper/2012.04357/) (2020)
- [Topology Distillation for Recommender System](https://mlsanity.com/paper/2106.08700/) (2021)
- [GenQREnsemble: Zero-Shot LLM Ensemble Prompting for Generative Query Reformulation](https://mlsanity.com/paper/2404.03746/) (2024)
- [Distillation from Heterogeneous Models for Top-K Recommendation](https://mlsanity.com/paper/2303.01130/) (2023)
- [Improving Retrieval in Theme-specific Applications using a Corpus Topical Taxonomy](https://mlsanity.com/paper/2403.04160/) (2024)
- [TWOLAR: a TWO-step LLM-Augmented distillation method for passage Reranking](https://mlsanity.com/paper/2403.17759/) (2024)
- [Dual Correction Strategy for Ranking Distillation in Top-N Recommender System](https://mlsanity.com/paper/2109.03459/) (2021)
- [MvFS: Multi-view Feature Selection for Recommender System](https://mlsanity.com/paper/2309.02064/) (2023)
- [Pre-training with Large Language Model-based Document Expansion for Dense Passage Retrieval](https://mlsanity.com/paper/2308.08285/) (2023)
- [Continual Collaborative Distillation for Recommender System](https://mlsanity.com/paper/2405.19046/) (2024)
- [Consensus Learning from Heterogeneous Objectives for One-Class Collaborative Filtering](https://mlsanity.com/paper/2202.13140/) (2022)

## Built on by (most cited first, 6 of 6)

- [Taxonomy-guided Semantic Indexing for Academic Paper Search](https://mlsanity.com/paper/2410.19218/) (2024)
- [Reference-Aligned Retrieval-Augmented Question Answering over Heterogeneous Proprietary Documents](https://mlsanity.com/paper/2502.19596/) (2025)
- [IRA: Adaptive Interest-aware Representation and Alignment for Personalized Multi-interest Retrieval](https://mlsanity.com/paper/2504.17529/) (2025)
- [Fine-tuning Small Language Models as Efficient Enterprise Search Relevance Labelers](https://mlsanity.com/paper/2601.03211/) (2026)
- [Scaling Search Relevance: Augmenting App Store Ranking with LLM-Generated Judgments](https://mlsanity.com/paper/2602.23234/) (2026)
- [PETRA: Transforming Web Text for Petroleum-Engineering Domain Adaptation](https://mlsanity.com/paper/2606.24346/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) excel at understanding the semantic relationships between queries and documents, even with lengthy and complex long-tail queries. These queries are challenging for feedback-based rankings due to sparse user engagement and limited feedback, making LLMs' ranking ability highly valuable. However, the large size and slow inference of LLMs necessitate the development of smaller, more efficient models (sLLMs). Recently, integrating ranking label generation into distillation techniques has become crucial, but existing methods underutilize LLMs' capabilities and are cumbersome. Our research, RRADistill: Re-Ranking Ability Distillation, propose an efficient label generation pipeline and novel sLLM training methods for both encoder and decoder models. We introduce an encoder-based method using a Term Control Layer to capture term matching signals and a decoder-based model with a ranking layer for enhanced understanding. A/B testing on a Korean-based search platform, validates the effectiveness of our approach in improving re-ranking for long-tail queries.