mlsanityData to 14 Oct 2026

Multi-Faceted Self-Consistent Preference Alignment for Query Rewriting in Conversational Search

arXiv 2604.06771 ↗First posted 8 Apr 2026cs.CL cs.AIcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Preference alignment for recommenders and RAG

Abstract (arXiv)

Conversational Query Rewriting (CQR) aims to rewrite ambiguous queries to achieve more efficient conversational search. Early studies have predominantly focused on the rewriting in isolation, ignoring the feedback from query rewrite, passage retrieval and response generation in the rewriting process. To address this issue, we propose Multi-Faceted Self-Consistent Preference Aligned CQR (MSPA-CQR). Specifically, we first construct self-consistent preference alignment data from three dimensions (rewriting, retrieval, and response) to generate more diverse rewritten queries. Then we propose prefix guided multi-faceted direct preference optimization to learn preference information from three different dimensions. The experimental results show that our MSPA-CQR is effective in both in- and out-of-distribution scenarios.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 19 of its 32 references that are in the map. First posted on arXiv 8 Apr 2026.

Builds on

19 of 32 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Adaptive Query Rewriting: Aligning Rewriters through Marginal Probability of Conversational Answers16 Jun
  2. 2024Ask Optimal Questions: Aligning Large Language Models with Retriever's Preference in Conversation19 Feb

Other references · 17

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2017Billion-scale similarity search with GPUscited by 2,497
  4. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  5. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 517
  6. 2023Fine-Tuning LLaMA for Multi-Stage Text Retrievalcited by 272
  7. 2020Open-Domain Question Answering Goes Conversational via Question Rewritingcited by 148
  8. 2021TopiOCQA: Open-domain Conversational Question Answering with Topic Switchingcited by 96
  9. 2023Large Language Models Know Your Contextual Search Intent: A Prompting Framework for Conversational Searchcited by 75
  10. 2023Enhancing Conversational Search: Large Language Model-Aided Informative Query Rewritingcited by 69
  11. 2021CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learningcited by 68
  12. 2020TREC CAsT 2019: The Conversational Assistance Track Overviewcited by 64
5 more
  1. 2023ConvGQR: Generative Query Reformulation for Conversational Searchcited by 59
  2. 2020Conversational Question Reformulation via Sequence-to-Sequence Architectures and Pretrained Language Modelscited by 45
  3. 2024CHIQ: Contextual History Enhancement for Improving Query Rewriting in Conversational Searchcited by 29
  4. 2023IterCQR: Iterative Conversational Query Reformulation with Retrieval Guidancecited by 20
  5. 2024AdaCQR: Enhancing Query Reformulation for Conversational Search via Sparse and Dense Retrieval Alignmentcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Factorized Hypothesis Search for Evidence-to-Taxonomy Retrievalcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025References Indeed Matter? Reference-Free Preference Optimization for Conversational Query Reformulation
  2. 2025ConvSearch-R1: Enhancing Query Reformulation for Conversational Search with Reasoning via Reinforcement Learning
  3. 2024Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generation
  4. 2026RIMS: Preference Optimization via Smoothed Multi-pair Aggregation for Small-Scale LLM Retrieval-Augmented Generation
  5. 2025From Clicks to Preference: A Multi-stage Alignment Framework for Generative Query Suggestion in Conversational System
  6. 2026CompCQR: Compositional Query Generation for Training-Free Conversational Search
  7. 2025AdaRewriter: Unleashing the Power of Prompting-based Conversational Query Reformulation via Test-Time Adaptation
  8. 2026RAC: Retrieval-Augmented Clarification for Faithful Conversational Search