mlsanityData to 14 Oct 2026

RAGferee: Building Contextual Reward Models for Retrieval-Augmented Generation

arXiv 2509.26011 ↗First posted 30 Sep 2025cs.CLcited by 1

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

also in 1 more
Abstract (arXiv)

Existing Reward Models (RMs), typically trained on general preference data, struggle in Retrieval Augmented Generation (RAG) settings, which require judging responses for faithfulness to retrieved context, relevance to the user query, appropriate refusals when context is insufficient, completeness and conciseness of information. To address the lack of publicly available RAG-centric preference datasets and specialised RMs, we introduce RAGferee, a methodology that repurposes question-answering (QA) datasets into preference pairs that prioritise groundedness over stylistic features, enabling the training of contextual RMs better suited to judging RAG responses. Using RAGferee, we curate a small preference dataset of 4K samples and fine-tune RMs ranging from 7B to 24B parameters. Our RAG-centric RMs achieve state-of-the-art performance on ContextualJudgeBench, surpassing existing 70B+ RMs trained on much larger (up to 2.4M samples) general corpora, with an absolute improvement of +15.5%.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 41 of its 71 references that are in the map. First posted on arXiv 30 Sep 2025.

Builds on

41 of 71 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment18 Dec
  2. 2024GroUSE: A Benchmark to Evaluate Evaluators in Grounded Question Answering10 Sep
  3. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents16 Apr

Other references · 38

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  8. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  9. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  10. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  11. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  12. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
26 more
  1. 2024A Survey on LLM-as-a-Judgecited by 1,423
  2. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  3. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  4. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  5. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 722
  6. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 575
  7. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 496
  8. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
  9. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 446
  10. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 368
  11. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 346
  12. 2024Skywork-Reward: Bag of Tricks for Reward Modeling in LLMscited by 327
  13. 2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 316
  14. 2023A Rank Stabilization Scaling Factor for Fine-Tuning with LoRAcited by 243
  15. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 226
  16. 2023FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Setscited by 201
  17. 2020A Survey of Evaluation Metrics Used for NLG Systemscited by 164
  18. 2024Generative Reward Modelscited by 148
  19. 2024Regularizing Hidden States Enables Learning Generalizable Reward Model for LLMscited by 142
  20. 2024VERISCORE: Evaluating the factuality of verifiable claims in long-form text generationcited by 117
  21. 2024OffsetBias: Leveraging Debiased Data for Tuning Evaluatorscited by 94
  22. 2025Learning to Plan & Reason for Evaluation with Thinking-LLM-as-a-Judgecited by 78
  23. 2024Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluationcited by 74
  24. 2024Self-Generated Critiques Boost Reward Modeling for Language Modelscited by 71
  25. 2024Beyond Scalar Reward Model: Learning Generative Judge from Preference Datacited by 32
  26. 2025Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settingscited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Building Legal Reward Models for Grounding and Abstentioncited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Reward-RAG: Enhancing RAG with Reward Driven Supervision
  2. 2024RichRAG: Crafting Rich Responses for Multi-faceted Queries in Retrieval-Augmented Generation
  3. 2025Enhancing Retrieval-Augmented Generation: A Study of Best Practices
  4. 2024Bridging the Preference Gap between Retrievers and LLMs
  5. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation
  6. 2025Distilling a Small Utility-Based Passage Selector to Enhance Retrieval-Augmented Generation
  7. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers
  8. 2024Toward Optimal Search and Retrieval for RAG