mlsanityData to 7 Oct 2026

RECTIFY: An Interactive Workbench for Post-Evaluation RAG Diagnosis, Repair, and Verification

arXiv 2609.16764 ↗First posted 15 Sep 2026cs.SEcited by 0

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) evaluators can identify failures such as weak retrieval, poor grounding, incomplete answers, and unsupported generation, but they rarely help developers decide what to repair next. We present RECTIFY, an interactive Streamlit workbench that turns evaluated RAG cases into auditable repair workflows. RECTIFY filters cases that do not require repair, routes remaining failures into actionable families and finegrained repair slices, and generates editable repair cards that developers can approve, reject, or verify through sandbox reruns. On a controlled RAG benchmark, RECTIFY surfaces interpretable failure profiles across BM25, dense, and hybrid retrieval: BM25 mainly triggers noisy-retrieval repairs, while dense and hybrid retrieval leave smaller sets of multi-part underretrieval and underused-evidence cases. Additional analyses show that pre-filtering reduces unnecessary repair candidates and that slicelevel routing yields more targeted repair cards than broad family-level diagnosis. RECTIFY is publicly available as an open-source Streamlit workbench 1 for helping developers turn evaluation results into inspectable repair decisions.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 11 of its 17 references that are in the map. First posted on arXiv 15 Sep 2026.

Builds on

11 of 17 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025RAGXplain: From Explainable Evaluation to Actionable Guidance of RAG Pipelines18 May
  2. 2024RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systems25 Jun
  3. 2024Evaluation of Retrieval-Augmented Generation: A Survey13 May
  4. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems16 Nov

Other references · 7

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  4. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  5. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 272
  6. 2024Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answeringcited by 95
  7. 2024CBR-RAG: Case-Based Reasoning for Retrieval Augmented Generation in LLMs for Legal Question Answeringcited by 70

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Diagnosing and Repairing Factual Errors in RAG under Budget Constraints
  2. 2026What Would Fix This RAG Failure? Auditing Counterfactual Response with Paired Evidence Interventions
  3. 2025RAGVUE: A Diagnostic View for Explainable and Automated Evaluation of Retrieval-Augmented Generation
  4. 2026Case-Aware LLM-as-a-Judge Evaluation for Enterprise-Scale RAG Systems
  5. 2026EviACT: An Evidence-to-Action Framework for Agentic Program Repair
  6. 2026BLAgent: Agentic RAG for File-Level Bug Localization
  7. 2026Debugging the Debuggers: Failure-Anchored Structured Recovery for Software Engineering Agents
  8. 2025FeedbackEval: A Benchmark for Evaluating Large Language Models in Feedback-Driven Code Repair Tasks