mlsanityData to 7 Oct 2026

ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agents

arXiv 2606.18037 ↗First posted 16 Jun 2026cs.AI cs.CL cs.MAcited by 1

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Tool-using LLM agents increasingly use the Model Context Protocol (MCP) to answer from heterogeneous evidence sources, including search, APIs, databases, clinical records, and formulary tools. Standard factuality metrics usually test whether an answer is supported by pooled evidence, missing a provenance-sensitive failure mode: a claim may be supported somewhere while being attributed to the wrong source. We call this cross-source conflation. We introduce ProvenanceGuard, a source-aware verifier for MCP-grounded answers. It consumes captured MCP traces with stable tool IDs, source IDs, and raw outputs; decomposes answers into atomic claims; routes claims to source-specific evidence; checks support with NLI and a token-alignment proxy; compares stated attribution with the routed source; and returns per-claim verdicts plus an answer-level allow/block decision. Blocked answers can be repaired with retrieval-augmented answer revision and re-verified. We evaluate on 281 medical-domain MCP-agent traces. A 266-trace adjudicated subset yields 2,325 LLM-assisted claim labels split by trace; 361 held-out labels are human-verified. On the 40-trace held-out split, ProvenanceGuard achieves block F1 0.802 and source accuracy 0.858 over 260 source-eligible claims, outperforming source-blind baselines that do not emit claim-to-source IDs. On a harder multi-source benchmark it reaches block F1 0.846, while source-plus-relation accuracy drops to 0.229, showing that exact source ownership remains difficult with semantically close sources. Repair-and-reverify resolves all blocked answers in the full trace set, often via conservative fallback. In 50 controlled clinical conflation probes, ProvenanceGuard detects all injected attribution swaps with no retained wrong attribution. These results show that source attribution is an independent axis for factuality verification in MCP-based agents.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 21 of its 29 references that are in the map. First posted on arXiv 16 Jun 2026.

Builds on

21 of 29 references.

References · 21

  1. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  2. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  3. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  4. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 408
  5. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 402
  6. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  7. 2023AlignScore: Evaluating Factual Consistency with a Unified Alignment Functioncited by 284
  8. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
  9. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 251
  10. 2020Evaluating Factuality in Generation with Dependency-level Entailmentcited by 122
  11. 2024VERISCORE: Evaluating the factuality of verifiable claims in long-form text generationcited by 114
  12. 2022Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Modelscited by 108
9 more
  1. 2023Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedbackcited by 71
  2. 2023Automatic Evaluation of Attribution by Large Language Modelscited by 70
  3. 2025FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generationcited by 27
  4. 2024Localizing Factual Inconsistencies in Attributable Text Generationcited by 13
  5. 2025LAQuer: Localized Attribution Queries in Content-grounded Generationcited by 13
  6. 2025GenerationPrograms: Fine-grained Attribution with Executable Programscited by 4
  7. 2024RAGulator: Lightweight Out-of-Context Detectors for Grounded Text Generationcited by 1
  8. 2025PrefixNLI: Detecting Factual Inconsistencies as Soon as They Arisecited by 1
  9. 2026Atomic Information Flow: A Network Flow Model for Tool Attributions in RAG Systemscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Engineering Reliable Commit Gates for Agentic AI: Cost-Aware Verification Portfolios under Common-Mode Data Failurescited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents
  2. 2026PARSE: Provenance-Aware Retrieval Sanitization for Professional Domain LLM Agents
  3. 2026EAVer: Long-Form Factuality Verification as an End-to-End Agentic Policy
  4. 2026Auditing Provenance Sensitivity in LLM Agent Action Selection
  5. 2026DRNOISE: Benchmarking Deep Research Agents in Misleading Evidence Environments
  6. 2025Veri-R1: Toward Precise and Faithful Claim Verification via Online Reinforcement Learning
  7. 2026Multi-Channel Mitigation of Source-Trust Shortcuts in Fact-Checking RL Agents
  8. 2026MedRAGChecker: Claim-Level Verification for Biomedical Retrieval-Augmented Generation