# Retrieval-Feedback-Driven Distillation and Preference Alignment for Efficient LLM-based Query Expansion

arXiv: https://arxiv.org/abs/2603.13776 · HTML: https://mlsanity.com/paper/2603.13776/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 17 of its 41 references that are in the map.
- First posted on arXiv 14 Mar 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Preference alignment for recommenders and RAG](https://mlsanity.com/alignment/preference-alignment-for-recommenders-and-rag/)

## Builds on (most cited first, 17 of 17)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Large Language Models for Information Retrieval: A Survey](https://mlsanity.com/paper/2308.07107/) (2023)
- [Overview of the TREC 2019 deep learning track](https://mlsanity.com/paper/2003.07820/) (2020)
- [How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition](https://mlsanity.com/paper/2310.05492/) (2023)
- [Efficient multi-prompt evaluation of LLMs](https://mlsanity.com/paper/2405.17202/) (2024)
- [BERT-QE: Contextualized Query Expansion for Document Re-ranking](https://mlsanity.com/paper/2009.07258/) (2020)
- [Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latency](https://mlsanity.com/paper/2504.03360/) (2025)
- [MILL: Mutual Verification with Large Language Models for Zero-Shot Query Expansion](https://mlsanity.com/paper/2310.19056/) (2023)
- [ExpandR: Teaching Dense Retrievers Beyond Queries with LLM Guidance](https://mlsanity.com/paper/2502.17057/) (2025)
- [Generative Question Refinement with Deep Reinforcement Learning in Retrieval-based QA System](https://mlsanity.com/paper/1908.05604/) (2019)
- [Domain Adaptation for Dense Retrieval and Conversational Dense Retrieval through Self-Supervision by Meticulous Pseudo-Relevance Labeling](https://mlsanity.com/paper/2403.08970/) (2024)
- [Query Expansion in the Age of Pre-trained and Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2509.07794/) (2025)
- [The Power of Selecting Key Blocks with Local Pre-ranking for Long Document Information Retrieval](https://mlsanity.com/paper/2111.09852/) (2021)
- [EviRerank: Adaptive Evidence Construction for Long-Document LLM Reranking](https://mlsanity.com/paper/2411.06254/) (2024)
- [Efficient Long-Document Reranking via Block-Level Embeddings and Top-k Interaction Refinement](https://mlsanity.com/paper/2501.17039/) (2025)

## Built on by (most cited first, 1 of 1)

- [Bounded Behavioral Indistinguishability for Black-Box LLM Distillation](https://mlsanity.com/paper/2605.30448/) (2026)

## Abstract (arXiv)

Large language models have recently enabled a generative paradigm for query expansion, but their high inference cost makes direct deployment difficult in practical retrieval systems. To address this issue, a retrieval-feedback-driven distillation and preference-alignment framework is proposed to transfer retrieval-friendly expansion behavior from a strong teacher model to a compact student model. Rather than relying on few-shot exemplars at inference time, the framework first leverages two complementary types of teacher-generated expansions, produced under zero-shot and few-shot prompting conditions, as supervision signals for distillation and as candidate pools for preference construction. A retrieval-metric-driven strategy is then introduced to automatically form chosen/rejected expansion pairs according to nDCG@10 differences, and Direct Preference Optimization is applied to explicitly align generation preferences with retrieval objectives. Experiments on TREC DL19/20/21 and MIRACL-zh show that the proposed approach preserves strong retrieval effectiveness while substantially reducing inference cost. In particular, the distilled Qwen3-4B model reaches about 97% of the teacher (DeepSeek-685B) model's nDCG@10 performance on DL19, and remains effective on the Chinese MIRACL-zh benchmark, demonstrating strong practicality across both English and Chinese retrieval settings.