AgentProv: Auditing Agentic LLM API Providers via Tool-use Policy Probes

arXiv 2609.00052 ↗First posted 30 Aug 2026cs.CR cs.CL cs.LGcited by 0

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Commercial LLM APIs advertise a specific foundation model, but the served backbone may be silently substituted, quantized, or wrapped, for example to save deployment costs. All existing audits decide backbone identity from the text-output channel, which is structurally fragile for agentic APIs because modern serving stacks (OpenAI, Anthropic, Gemini, Cloudflare Workers AI, LangGraph) discard text and expose only structured actions when the model calls a tool, and provider-injected system prompts can distort text distributions enough that text-channel tests falsely accuse honest providers of substituting the claimed model. We observe that recent agentic post-training internalizes tool-use directly into the weights, opening a new audit channel that the serving stack still exposes and that is largely invariant to deployment context. We introduce Agentic Provenance (AgentProv), the first action-based identity audit for agentic LLM APIs: AgentProv fingerprints a deployed model through its categorical tool-call distribution and decides identity via an MMD permutation test. AgentProv catches every substituted model (100% on 630 evaluated checkpoint pairs), while holding the false-positive rate under system-prompt injection at 7% (vs. 67% for MET and 53% for RUT). On third-party API endpoints, AgentProv's disagreements with MET are consistent with an independent token-count side-channel that detects provider-injected system prompts.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 19 of its 33 references that are in the map. First posted on arXiv 30 Aug 2026.

Builds on

19 of 33 references.

Closest ancestors · 1

Same direction, newest first

2026On Protecting Agentic Systems' Intellectual Property via Watermarking9 Feb

Other references · 18

2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 392
2024LLMmap: Fingerprinting For Large Language Modelscited by 58
2024Model Equality Testing: Which Model Is This API Serving?cited by 47
2024Explanation as a Watermark: Towards Harmless and Multi-bit Model Ownership Verification via Watermarking Feature Attributioncited by 40
2025Are You Getting What You Pay For? Auditing Model Substitution in LLM APIscited by 38
2024TRAP: Targeted Random Adversarial Prompt Honeypot for Black-Box Identificationcited by 36
2024Your Large Language Models Are Leaving Fingerprintscited by 33
2024Hide and Seek: Fingerprinting Large Language Models with Evolutionary Learningcited by 18

6 more

2025Scalable Fingerprinting of Large Language Modelscited by 14
2025FIT-Print: Towards False-claim-resistant Model Ownership Verification via Targeted Fingerprintcited by 10
2025RAFP: Identifying LLM Lineages via Rare-Region Fingerprintscited by 8
2025DuFFin: A Dual-Level Fingerprinting Framework for LLMs IP Protectioncited by 8
2026Real Money, Fake Models: Deceptive Model Claims in Shadow APIscited by 8
2026Your "Pro" LLM Subscription May Actually Be "Free": Exposing Fingerprint Spoofing Risks in LLM Inference Servicescited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way