mlsanityData to 14 Oct 2026

A Triple-Robustness Analysis of Retrieval-Augmented Generation for Multi-Hop Requirements Traceability

arXiv 2608.00705 ↗First posted 1 Aug 2026cs.IR cs.CLcited by 0

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Reported verdicts on GraphRAG versus vector RAG disagree, and the evidence is typically tied to a single corpus, embedder, and judge -- and, we show, to where citation quality is measured. We present a triple-robustness analysis that holds a five-pipeline architecture matrix fixed and varies embedder (local e5-small vs. Azure text-embedding-3-small), corpus (DO-178C typed-edge requirements vs. Wikipedia paragraph chains via MuSiQue), and judge (paired GPT-5.4 x GPT-4.1 on both corpora), over 2x4,440 main-matrix runs, 600 cross-corpus runs, and over 5,000 faithfulness judgments. (C2a) GraphRAG's graph walk floods the context window at precision 0.12-0.23, but the synthesizer cites selectively at precision 0.48-0.65; scoring the retrieved set as the attribution set inverts the architecture ranking, which reconciles part of the disagreement in prior reports. (C1) Answer-level citation winners are corpus- and stratum-conditional but embedder-robust: GraphRAG ties vanilla on short-hop DO-178C queries and wins every MuSiQue stratum, while agentic pipelines lead only on 3+-hop requirements queries. (C2b) Faithfulness is corpus-conditional: on DO-178C it declines with hop distance (trend p<0.05 in three of four judge x embedder combinations); on Wikipedia chains neither judge shows a collapse. (C3) Single-judge LLM faithfulness is fragile to retrieval state: GPT-5.4's self-kappa across embedders is 0.137 (41% verdict change) against a same-day test-retest floor of 0.76, and re-judging frozen inputs eleven weeks later gives kappa <= 0.14 for both judges. A learned router on dense embeddings alone reaches macro-F1 0.86 on hop classification (C4). We argue that RAG architecture claims should be tested at this level of robustness -- including robustness to the citation-measurement point -- before they are trusted.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 10 of its 27 references that are in the map. First posted on arXiv 1 Aug 2026.

Builds on

10 of 27 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity21 Mar
  2. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection17 Oct

Other references · 8

  1. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  2. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  3. 2024Self-Preference Bias in LLM-as-a-Judgecited by 258
  4. 2024MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queriescited by 257
  5. 2025RAG vs. GraphRAG: A Systematic Evaluation and Key Insightscited by 53
  6. 2024Probing-RAG: Self-Probing to Guide Language Models in Selective Document Retrievalcited by 29
  7. 2025When +1% Is Not Enough: A Paired Bootstrap Protocol for Evaluating Small Improvementscited by 7
  8. 2024Leveraging Graph-RAG and Prompt Engineering to Enhance LLM-Based Automated Requirement Traceability and Compliance Checkscited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Universal Pathologies, Conditional Consequences: A Triple-Robustness Analysis of RAG for Multi-Hop Traceability
  2. 2024Towards Fine-Grained Citation Evaluation in Generated Text: A Comparative Analysis of Faithfulness Metrics
  3. 2024A Comparative Analysis of Faithfulness Metrics and Humans in Citation Evaluation
  4. 2025RARE: Retrieval-Aware Robustness Evaluation for Retrieval-Augmented Generation Systems
  5. 2026The Attribution-Compression Frontier in Retrieval-Augmented Generation
  6. 2026DS@GT ARC at LongEval: Citation Integrity and Factual Grounding in Scientific QA
  7. 2026EvidenT: Building Trustworthy Enterprise Assistants through Evidence Groundedness and Traceability
  8. 2026TRIAGE: Trustworthy Retrieval Instrumentation And Graph Evaluation