Universal Pathologies, Conditional Consequences: A Triple-Robustness Analysis of RAG for Multi-Hop Traceability
Not on the map. Why?
Abstract (arXiv)
GraphRAG underperforms vector RAG on citation precision in many reports, but where and why have remained corpus-bound. We present a triple-robustness analysis that holds the retrieval architecture fixed and varies three orthogonal axes embedder (local e5-small -> Azure text-embedding-3-small), corpus (DO-178C typed-edge requirements -> Wikipedia paragraph chains via MuSiQue), and judge (paired GPT-5.4 x GPT-4.1) across 4,440 main-matrix runs, 600 cross-corpus runs, and 1,200 paired faithfulness judgments. (C2a) Over-citation is architecturally universal: GraphRAG emits 11-15 IDs per answer at citation precision 0.12-0.23 and retrieval recall 0.68-0.87 across all three settings. (C2b) Its faithfulness consequence is corpus-conditional: in typed-edge DO-178C, GraphRAG faithfulness collapses 74%->40% across hops; on Wikipedia chains the same pipeline rises 42%->58% because over-cited paragraphs remain topically supporting. (C1) Stratum-conditional winners are corpus-conditional but embedder-robust: vanilla wins 2-hop on DO-178C, GraphRAG wins 2-hop on MuSiQue, identical under either embedder. (C3) Single-judge LLM faithfulness is fragile to retrieval state: same-judge self-kappa across embedders is 0.137 for GPT-5.4 (verdict change on 41% of items). A learned router on dense embeddings alone reaches macro-F1 0.86 on hop classification (C4). We argue triple-robustness is the minimum bar for trustworthy RAG architecture claims.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 26 references that are in the map. First posted on arXiv 20 May 2026.
Builds on
10 of 26 references.
References · 10
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
- 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
- 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
- 2023Enabling Large Language Models to Generate Text with Citationscited by 595
- 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
- 2024Self-Preference Bias in LLM-as-a-Judgecited by 258
- 2024MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queriescited by 257
- 2025RAG vs. GraphRAG: A Systematic Evaluation and Key Insightscited by 53
- 2024Probing-RAG: Self-Probing to Guide Language Models in Selective Document Retrievalcited by 29
- 2025When +1% Is Not Enough: A Paired Bootstrap Protocol for Evaluating Small Improvementscited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026A Triple-Robustness Analysis of Retrieval-Augmented Generation for Multi-Hop Requirements Traceability
- 2026Why Neighborhoods Matter: Traversal Context and Provenance in Agentic GraphRAG
- 2024A Comparative Analysis of Faithfulness Metrics and Humans in Citation Evaluation
- 2024Towards Fine-Grained Citation Evaluation in Generated Text: A Comparative Analysis of Faithfulness Metrics
- 2025Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievals
- 2025CiteCheck: Towards Accurate Citation Faithfulness Detection
- 2026FACTUM: Mechanistic Detection of Citation Hallucination in Long-Form RAG
- 2026WildGraphBench: Benchmarking GraphRAG with Wild-Source Corpora