mlsanityData to 7 Oct 2026

LayerRAG-Bench: A Cross-Layer Reliability Benchmark for Agentic Retrieval-Augmented Generation

arXiv 2607.27353 ↗First posted 29 Jul 2026cs.CLcited by 0

On the map: Retrieval-augmented generation → Agentic RAG, agents and memory

also in 1 more
Abstract (arXiv)

Agentic retrieval-augmented generation systems can produce answers that appear grounded while failing at the evidence, tool-contract, authorization, or session-state layer. We introduce LayerRAG-Bench, a controlled cross-layer reliability benchmark with 8 enterprise domains, 240 tasks, 9 fault scenarios, 2 contract modes, and 38,880 live task-level records across nine models from OpenAI, Anthropic, and Gemini. Schema normalization raises schema-drift success from 0.000 to 0.913, but stale evidence, missing tool output, denied permissions, and wrong-session context are not recovered by schema normalization. Groundedness-only evaluation also produces substantial false positives under stale and wrong-session evidence. These results support a layer-specific evaluation principle: a reliability intervention should be credited for repairing its target layer without being mistaken for a universal fix.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation. Builds on 17 of its 18 references that are in the map. First posted on arXiv 29 Jul 2026.

Builds on

17 of 18 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Doctor-RAG: A Failure-Aware Repair Framework for Agentic Retrieval-Augmented Generation1 Apr
  2. 2026SoK: Agentic Retrieval-Augmented Generation (RAG): Taxonomy, Architectures, Evaluation, and Research Directions7 Mar
  3. 2025RAGCap-Bench: Benchmarking Capabilities of LLMs in Agentic Retrieval Augmented Generation Systems15 Oct

Other references · 14

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  5. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  6. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  7. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  8. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 408
  9. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 259
  10. 2024CRAG -- Comprehensive RAG Benchmarkcited by 94
  11. 2026ReliabilityBench: Evaluating LLM Agent Reliability Under Production-Like Stress Conditionscited by 31
  12. 2026EnterpriseRAG-Bench: A RAG Benchmark for Company Internal Knowledgecited by 7
2 more
  1. 2026AgenticRAGTracer: A Hop-Aware Benchmark for Diagnosing Multi-Step Retrieval Reasoning in Agentic RAGcited by 5
  2. 2026Overcoming the "Impracticality" of RAG: Proposing a Real-World Benchmark and Multi-Dimensional Diagnostic Frameworkcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026AgentJudgeBench: A Multi-Difficulty Benchmark for Evaluating LLM Judges on Agentic Tool-Calling
  2. 2026Before Reasoning Can Fail: Pre-Evidence Procedural Failures in Agentic RAG
  3. 2026Beyond Fluency: Toward Reliable Trajectories in Agentic IR
  4. 2026Orchestrating Specialized Agents for Trustworthy Enterprise RAG
  5. 2026When Failures Propagate: Causal Failure Attribution in Agentic Retrieval-Augmented Generation
  6. 2026The Double Measurement Confound in Agent Benchmarks: De-Scaffolding, Ground-Truth Scoring, and Reliability Beyond the Mean
  7. 2026Improving Retrieval-Augmented Generation without Taxonomy-based Error Categorization
  8. 2026Building Trustworthy Graph-Agentic RAG for Social Good: Architectures, Failure Propagation, and Assurance by Construction