mlsanityData to 14 Oct 2026

RaFe: Ranking Feedback Improves Query Rewriting for RAG

arXiv 2405.14431 ↗First posted 23 May 2024cs.CL cs.AI cs.IRcited by 49

On the map: Retrieval-augmented generation → Generation and context integration

Abstract (arXiv)

As Large Language Models (LLMs) and Retrieval Augmentation Generation (RAG) techniques have evolved, query rewriting has been widely incorporated into the RAG system for downstream tasks like open-domain QA. Many works have attempted to utilize small models with reinforcement learning rather than costly LLMs to improve query rewriting. However, current methods require annotations (e.g., labeled relevant documents or downstream answers) or predesigned rewards for feedback, which lack generalization, and fail to utilize signals tailored for query rewriting. In this paper, we propose ours, a framework for training query rewriting models free of annotations. By leveraging a publicly available reranker, ours~provides feedback aligned well with the rewriting objectives. Experimental results demonstrate that ours~can obtain better performance than baselines.

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 33 of its 52 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

33 of 52 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP28 Dec

Other references · 32

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  6. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  7. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  8. 2023A Survey of Large Language Modelscited by 2,576
  9. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  10. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  11. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  12. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
20 more
  1. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  2. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 372
  3. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  4. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 318
  5. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 246
  6. 2023Secrets of RLHF in Large Language Models Part I: PPOcited by 229
  7. 2024Secrets of RLHF in Large Language Models Part II: Reward Modelingcited by 171
  8. 2023RLTF: Reinforcement Learning from Unit Test Feedbackcited by 114
  9. 2023RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputscited by 105
  10. 2023Language Model Self-improvement by Reinforcement Learning Contemplationcited by 75
  11. 2023Enhancing Conversational Search: Large Language Model-Aided Informative Query Rewritingcited by 69
  12. 2021CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learningcited by 68
  13. 2023Large Language Model based Long-tail Query Rewriting in Taobao Searchcited by 62
  14. 2023ConvGQR: Generative Query Reformulation for Conversational Searchcited by 59
  15. 2017Task-Oriented Query Reformulation with Reinforcement Learningcited by 39
  16. 2016Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answeringcited by 35
  17. 2023Large Language Models are Strong Zero-Shot Retrievercited by 26
  18. 2022Reinforced Question Rewriting for Conversational Question Answeringcited by 19
  19. 2023Crystal: Introspective Reasoners Reinforced with Self-Feedbackcited by 15
  20. 2023Reasons to Reject? Aligning Language Models with Judgmentscited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 25; 2026: 14.

49 papers.

Sort:

By area:

  1. 2024Graph Retrieval-Augmented Generation: A Surveycited by 324
  2. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 101
  3. 2024Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Modelscited by 44
  4. 2024DMQR-RAG: Diverse Multi-Query Rewriting for RAGcited by 27
  5. 2024ChunkRAG: Novel LLM-Chunk Filtering Method for RAG Systemscited by 23
  6. 2024HyPA-RAG: A Hybrid Parameter Adaptive Retrieval-Augmented Generation System for AI Legal and Policy Applicationscited by 22
  7. 2025Collab-RAG: Boosting Retrieval-Augmented Generation for Complex Question Answering via White-Box and Black-Box LLM Collaborationcited by 20
  8. 2024OneGen: Efficient One-Pass Unified Generation and Retrieval for LLMscited by 14
  9. 2025Towards Omni-RAG: Comprehensive Retrieval-Augmented Generation for Large Language Models in Medical Applicationscited by 13
  10. 2025ExpandR: Teaching Dense Retrievers Beyond Queries with LLM Guidancecited by 10
  11. 2024AutoMIR: Effective Zero-Shot Medical Information Retrieval without Relevance Labelscited by 8
  12. 2025MIRIAD: Augmenting LLMs with millions of medical query-response pairscited by 8
37 more
  1. 2025Disambiguation in Conversational Question Answering in the Era of LLMs and Agents: A Surveycited by 6
  2. 2026Why Your Deep Research Agent Fails? On Hallucination Evaluation in Full Research Trajectorycited by 5
  3. 2025Automatic Input Rewriting Improves Translation with Large Language Modelscited by 4
  4. 2025KG-Infused RAG: Augmenting Corpus-Based RAG with External Knowledge Graphscited by 4
  5. 2025Leveraging LLM-Assisted Query Understanding for Live Retrieval-Augmented Generationcited by 4
  6. 2025How Good are LLM-based Rerankers? An Empirical Analysis of State-of-the-Art Reranking Modelscited by 4
  7. 2025Advancing Multi-Agent RAG Systems with Minimalist Reinforcement Learningcited by 3
  8. 2026Supervised Fine-Tuning versus Reinforcement Learning: A Study of Post-Training Methods for Large Language Modelscited by 3
  9. 2025Less Is More: Elevating RAG via Performance-Driven Context Compressioncited by 2
  10. 2025MuISQA: Multi-Intent Retrieval-Augmented Generation for Scientific Question Answeringcited by 2
  11. 2024A Survey of Query Optimization in Large Language Modelscited by 1
  12. 2025Foundations of GenIRcited by 1
  13. 2025Cognitive-Aligned Document Selection for Retrieval-augmented Generationcited by 1
  14. 2025Single LLM, Multiple Roles: A Unified Retrieval-Augmented Generation Framework Using Role-Specific Token Optimizationcited by 1
  15. 2025Read the Docs Before Rewriting: Equip Rewriter with Domain Knowledge via Continual Pre-trainingcited by 1
  16. 2025KG-CQR: Leveraging Structured Relation Representations in Knowledge Graphs for Contextual Query Retrievalcited by 1
  17. 2025Sherlock Your Queries: Learning to Ask the Right Questions for Dialogue-Based Retrievalcited by 1
  18. 2025Foundation of Intelligence: Review of Math Word Problems from Human Cognition Perspectivecited by 1
  19. 2025PICOs-RAG: PICO-supported Query Rewriting for Retrieval-Augmented Generation in Evidence-Based Medicinecited by 1
  20. 2025QueStER: Query Specification for Generative keyword-based Retrievalcited by 1
  21. 2026Long-Context Long-Form Question Answering for Legal Domaincited by 1
  22. 2026Help Without Being Asked: A Deployed Proactive Agent System for On-Call Support with Continuous Self-Improvementcited by 1
  23. 2026LLM-Oriented Information Retrieval: A Denoising-First Perspectivecited by 1
  24. 2024R^3AG: First Workshop on Refined and Reliable Retrieval Augmented Generationcited by 0
  25. 2025Retrieval-Augmented Generation Systems for Intellectual Property via Synthetic Multi-Angle Fine-tuningcited by 0
  26. 2025KARE-RAG: Knowledge-Aware Refinement and Enhancement for RAGcited by 0
  27. 2025Can Synthetic Query Rewrites Capture User Intent Better than Humans in Retrieval-Augmented Generation?cited by 0
  28. 2025Adaptive Data Flywheel: Applying MAPE Control Loops to AI Agent Improvementcited by 0
  29. 2026No One Size Fits All: QueryBandits for Hallucination Mitigationcited by 0
  30. 2026Masking or Mitigating? Deconstructing the Impact of Query Rewriting on Retriever Biases in RAGcited by 0
  31. 2026LeakDojo: Decoding the Leakage Threats of RAG Systemscited by 0
  32. 2026CoHyDE: Iterative Co-Training of LLM Rewriter & Dense Encoder for Tool Retrievalcited by 0
  33. 2026STORM: Stepwise Token Optimization with Reward-Guided Beam Searchcited by 0
  34. 2026Neighborhood-Aware Dual Biomedical Entity Linkingcited by 0
  35. 2026Does the Selected Object Reach the Reader? Auditing Identity Handoffs in Grounded Language-Model Pipelinescited by 0
  36. 2026Bayesian Test-Time Inference of Task-Aligned Similarity from Weak Interactive Feedbackcited by 0
  37. 2026GraphRAG-LTM: A Layered Threat Modeling Framework for Attacks on Graph Retrieval-Augmented Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MaFeRw: Query Rewriting with Multi-Aspect Feedbacks for Retrieval-Augmented Large Language Models
  2. 2026Better Together: Complementary Query Rewriting Under a Strong RAG Baseline
  3. 2023Query Rewriting for Retrieval-Augmented Large Language Models
  4. 2025DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generation
  5. 2026Not All Queries Need Rewriting: When Prompt-Only LLM Refinement Helps and Hurts Dense Retrieval
  6. 2025Annotation-Free Reinforcement Learning Query Rewriting via Verifiable Search Reward
  7. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation
  8. 2026Optimizing RAG Rerankers with LLM Feedback via Reinforcement Learning