mlsanityData to 7 Oct 2026

Retrieval-Feedback-Driven Distillation and Preference Alignment for Efficient LLM-based Query Expansion

arXiv 2603.13776 ↗First posted 14 Mar 2026cs.IR cs.AIcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Preference alignment for recommenders and RAG

Abstract (arXiv)

Large language models have recently enabled a generative paradigm for query expansion, but their high inference cost makes direct deployment difficult in practical retrieval systems. To address this issue, a retrieval-feedback-driven distillation and preference-alignment framework is proposed to transfer retrieval-friendly expansion behavior from a strong teacher model to a compact student model. Rather than relying on few-shot exemplars at inference time, the framework first leverages two complementary types of teacher-generated expansions, produced under zero-shot and few-shot prompting conditions, as supervision signals for distillation and as candidate pools for preference construction. A retrieval-metric-driven strategy is then introduced to automatically form chosen/rejected expansion pairs according to nDCG@10 differences, and Direct Preference Optimization is applied to explicitly align generation preferences with retrieval objectives. Experiments on TREC DL19/20/21 and MIRACL-zh show that the proposed approach preserves strong retrieval effectiveness while substantially reducing inference cost. In particular, the distilled Qwen3-4B model reaches about 97% of the teacher (DeepSeek-685B) model's nDCG@10 performance on DL19, and remains effective on the Chinese MIRACL-zh benchmark, demonstrating strong practicality across both English and Chinese retrieval settings.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 17 of its 41 references that are in the map. First posted on arXiv 14 Mar 2026.

Builds on

17 of 41 references.

References · 17

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  4. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  5. 2020Overview of the TREC 2019 deep learning trackcited by 282
  6. 2023How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Compositioncited by 222
  7. 2024Efficient multi-prompt evaluation of LLMscited by 72
  8. 2020BERT-QE: Contextualized Query Expansion for Document Re-rankingcited by 40
  9. 2025Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latencycited by 32
  10. 2023MILL: Mutual Verification with Large Language Models for Zero-Shot Query Expansioncited by 22
  11. 2025ExpandR: Teaching Dense Retrievers Beyond Queries with LLM Guidancecited by 10
  12. 2019Generative Question Refinement with Deep Reinforcement Learning in Retrieval-based QA Systemcited by 6
5 more
  1. 2024Domain Adaptation for Dense Retrieval and Conversational Dense Retrieval through Self-Supervision by Meticulous Pseudo-Relevance Labelingcited by 5
  2. 2025Query Expansion in the Age of Pre-trained and Large Language Models: A Comprehensive Surveycited by 5
  3. 2021The Power of Selecting Key Blocks with Local Pre-ranking for Long Document Information Retrievalcited by 3
  4. 2024EviRerank: Adaptive Evidence Construction for Long-Document LLM Rerankingcited by 3
  5. 2025Efficient Long-Document Reranking via Block-Level Embeddings and Top-k Interaction Refinementcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Bounded Behavioral Indistinguishability for Black-Box LLM Distillationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023Query Expansion by Prompting Large Language Models
  2. 2024Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generation
  3. 2025GainRAG: Preference Alignment in Retrieval-Augmented Generation through Gain Signal Synthesis
  4. 2024Fine-Grained Guidance for Retrievers: Leveraging LLMs' Feedback in Retrieval-Augmented Generation
  5. 2025Enhancing Retrieval-Augmented Generation with Two-Stage Retrieval: FlashRank Reranking and Query Expansion
  6. 2026RIMS: Preference Optimization via Smoothed Multi-pair Aggregation for Small-Scale LLM Retrieval-Augmented Generation
  7. 2025RoseRAG: Robust Retrieval-augmented Generation with Small-scale LLMs via Margin-aware Preference Optimization
  8. 2026Query Expansion Should Be Coordinated: Dense Expands, Sparse Anchors