mlsanityData to 7 Oct 2026

RAGWarrant: Evidence-Preserving Governance for RAG Policy Promotion Under Quality, Cost, Latency, and Risk Constraints

arXiv 2609.34179 ↗First posted 28 Sep 2026cs.AI cs.CL cs.LGcited by 0

On the map: Retrieval-augmented generation → Agentic RAG, agents and memory also fits: Legal and financial RAG

Abstract (arXiv)

Retrieval-augmented generation systems are extensively instrumented with metrics, benchmarks, traces, and automated judges, but these tools do not decide whether a proposed policy change is safe to release. We present RAGWarrant, an open-source promotion-control framework that treats deployment as a constrained evidence decision rather than a leaderboard choice. RAGWarrant normalizes evaluator outputs and operational telemetry, applies predeclared quality and hard-risk gates, assigns evidence-class claim ceilings, preserves negative outcomes, and emits auditable PROMOTE, BLOCK, REJECT, or INCONCLUSIVE decisions. We evaluate the framework across T2-RAGBench, MultiHop-RAG, CRAG, HotpotQA, synthetic reproduction, and bounded local generative experiments. On HotpotQA, operational savings were blocked because answer quality fell beyond the declared margin. A bounded CRAG study selected a lower-cost quality-tied policy, but related generative gains were unstable and a held-out guardrail failed closed. We claim an auditable promotion-control abstraction, not optimizer superiority, human validation, or production readiness. The tagged artifact reproduces from a fresh clone, runs as a hardened Docker job, accepts external evaluator exports, and verifies artifact integrity.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation. Builds on 12 of its 21 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

12 of 21 references.

References · 12

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  3. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  4. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  5. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 408
  6. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 307
  7. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 259
  8. 2024MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queriescited by 253
  9. 2024RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systemscited by 115
  10. 2024CRAG -- Comprehensive RAG Benchmarkcited by 94
  11. 2024RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generationcited by 59
  12. 2025T2-RAGBench: Text-and-Table Benchmark for Evaluating Retrieval-Augmented Generationcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026When Knowledge Is Not Free: Cost-Aware Evidence Selection in Retrieval-Augmented Generation
  2. 2026AGO AI Quality Gate: Evidence-First Release Decisions for Retrieval-Augmented Generation
  3. 2026The RAT: A Unified Bayesian Model for RAG Evaluation
  4. 2025Towards Transparent RAG: Fostering Evidence Traceability in LLM Generation via Reinforcement Learning
  5. 2026When Should Active RAG Retrieve? A Budget-Aware Evaluation of Utility, Calibration, and Cost
  6. 2025Advancing Risk and Quality Assurance: A RAG Chatbot for Improved Regulatory Compliance
  7. 2025RAGXplain: From Explainable Evaluation to Actionable Guidance of RAG Pipelines
  8. 2025RAG-Zeval: Towards Robust and Interpretable Evaluation on RAG Responses through End-to-End Rule-Guided Reasoning