mlsanityData to 7 Oct 2026

Reward-RAG: Enhancing RAG with Reward Driven Supervision

arXiv 2410.03780 ↗First posted 3 Oct 2024cs.CL cs.LGcited by 7

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

also in 1 more
Abstract (arXiv)

In this paper, we introduce Reward-RAG, a novel approach designed to enhance the Retrieval-Augmented Generation (RAG) model through Reward-Driven Supervision. Unlike previous RAG methodologies, which focus on training language models (LMs) to utilize external knowledge retrieved from external sources, our method adapts retrieval information to specific domains by employing CriticGPT to train a dedicated reward model. This reward model generates synthesized datasets for fine-tuning the RAG encoder, aligning its outputs more closely with human preferences. The versatility of our approach allows it to be effectively applied across various domains through domain-specific fine-tuning. We evaluate Reward-RAG on publicly available benchmarks from multiple domains, comparing it to state-of-the-art methods. Our experimental results demonstrate significant improvements in performance, highlighting the effectiveness of Reward-RAG in improving the relevance and quality of generated responses. These findings underscore the potential of integrating reward models with RAG to achieve superior outcomes in natural language generation tasks.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 88 of its 106 references that are in the map. First posted on arXiv 3 Oct 2024.

Builds on

88 of 106 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Reinforcement Learning for Optimizing RAG for Domain Chatbots10 Jan
  2. 2023Retrieval-Augmented Generation for Large Language Models: A Survey18 Dec

Other references · 86

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2017Proximal Policy Optimization Algorithmscited by 14,441
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  9. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  10. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  11. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  12. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
38 more
  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  9. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  10. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  11. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  12. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  13. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  14. 2020Learning to summarize from human feedbackcited by 2,562
  15. 2023A Survey of Large Language Modelscited by 2,537
  16. 2017Billion-scale similarity search with GPUscited by 2,477
  17. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  18. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  19. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  20. 2024Mixtral of Expertscited by 1,785
  21. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  22. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  23. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  24. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  25. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  26. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  27. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  28. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  29. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  30. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  31. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  32. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  33. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  34. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
  35. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  36. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  37. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  38. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 556

50 most cited of 86 All 86 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 1.

7 papers.

  1. 2025Knowing You Don't Know: Learning When to Continue Search in Multi-round RAG through Self-Practicingcited by 24
  2. 2025DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generationcited by 14
  3. 2025Can Knowledge-Graph-based Retrieval Augmented Generation Really Retrieve What You Need?cited by 8
  4. 2025A Comprehensive Graph Framework for Question Answering with Mode-Seeking Preference Alignmentcited by 5
  5. 2025Reinforced Informativeness Optimization for Long-Form Retrieval-Augmented Generationcited by 2
  6. 2025LMAR: Language Model Augmented Retriever for Domain-specific Knowledge Indexingcited by 1
  7. 2026Reinforcement Fine-Tuning for History-Aware Dense Retriever in RAGcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025RAGferee: Building Contextual Reward Models for Retrieval-Augmented Generation
  2. 2024RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment
  3. 2026Search-P1: Path-Centric Reward Shaping for Stable and Efficient Agentic RAG Training
  4. 2024RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewards
  5. 2025SKILL-RAG: Self-Knowledge Induced Learning and Filtering for Retrieval-Augmented Generation
  6. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning
  7. 2026CTRL-RAG: Contrastive Likelihood Reward Based Reinforcement Learning for Context-Faithful RAG Models
  8. 2024Retrieval-Augmented Generation for Natural Language Processing: A Survey