Who Is Behind the Harness? Fingerprinting LLMs through Agentic Behavior
On the map: LLM agents and tool use → Agent threats, guardrails and governance also fits: Software engineering coding agents
Abstract (arXiv)
LLMs increasingly operate through coding-agent harnesses that inspect repositories, invoke tools, and modify files. Substituting the model behind such an agent can therefore change security-relevant decisions, including whether it verifies changes or recovers safely from failures. Existing LLM fingerprints largely infer identity from direct text or token distributions. In coding agents, these signals are mediated by system instructions, controller logic, tools, and execution feedback, limiting their transfer. We present LIDAR (LLM Identification from Decisions and Actions at Runtime), an active black-box fingerprinting method for coding-agent execution. Three coding probe pairs expose post-edit verification, transient-failure recovery, and specification--test conflict resolution under controlled changes. LIDAR represents the resulting trajectories with complementary instance-level and distribution-level features and compares them with clean references using a lightweight probabilistic identifier. It requires no access to model weights, logits, or provider internals. Across 36 models from seven families and two agent harnesses, LIDAR achieves high Top-1 accuracy and MRR and outperforms four existing fingerprinting and API-auditing baselines. Ablations confirm that the two feature levels, all probe pairs, and their controlled variants contribute. These results show that agent execution behavior provides model-identity evidence beyond final outputs.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 25 of its 33 references that are in the map. First posted on arXiv 23 Sep 2026.
Builds on
25 of 33 references.
Closest ancestors · 7
Same direction, newest first
- 2026TRACE: A Two-Channel Robust Attribution Watermark via Complementary Embeddings for LLM-Agent Trajectories9 Jul
- 2026Black-Box Forensics for Conversational LLM Agents21 Jun
- 2026TrustedARI: Towards Trust-Native Agentic Routing Infrastructure for Agentic AI14 Jun
- 2026Sequential Behavioral Watermarking for LLM Agents11 May
- 2026On Protecting Agentic Systems' Intellectual Property via Watermarking9 Feb
- 2026AgentMark: Utility-Preserving Behavioral Watermarking for Agents5 Jan
- 2025Agent Guide: A Simple Agent Behavioral Watermarking Framework8 Apr
Other references · 18
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
- 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
- 2023A Watermark for Large Language Modelscited by 667
- 2025MiniMax-01: Scaling Foundation Models with Lightning Attentioncited by 157
- 2023REMARK-LLM: A Robust and Efficient Watermarking Framework for Generative Large Language Modelscited by 64
- 2024Model Equality Testing: Which Model Is This API Serving?cited by 44
- 2024Provably Robust Multi-bit Watermarking for AI-generated Textcited by 38
- 2025Are You Getting What You Pay For? Auditing Model Substitution in LLM APIscited by 35
- 2024TRAP: Targeted Random Adversarial Prompt Honeypot for Black-Box Identificationcited by 33
- 2024Watermarking Language Models for Many Adaptive Userscited by 16
- 2025Auditing Black-Box LLM APIs with a Rank-Based Uniformity Testcited by 16
- 2025SoK: Large Language Model Copyright Auditing via Fingerprintingcited by 14
6 more
- 2021Did the Model Change? Efficiently Assessing Machine Learning API Shiftscited by 13
- 2025Fingerprinting LLMs via Prompt Injectioncited by 8
- 2025CoTSRF: Utilize Chain of Thought as Stealthy and Robust Fingerprint of Large Language Modelscited by 7
- 2025Character-Level Perturbations Disrupt LLM Watermarkscited by 7
- 2026One Token Is Enough: Fingerprinting and Verifying Large Language Models from Single-Token Output Distributionscited by 6
- 2026Your "Pro" LLM Subscription May Actually Be "Free": Exposing Fingerprint Spoofing Risks in LLM Inference Servicescited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026TraceLab: Characterizing Coding Agent Workloads for LLM Serving
- 2026AgentProv: Auditing Agentic LLM API Providers via Tool-use Policy Probes
- 2025Exposing LLM User Privacy via Traffic Fingerprint Analysis: A Study of Privacy Risks in LLM Agent Interactions
- 2025Verifiability-First Agents: Provable Observability and Lightweight Audit Agents for Controlling Autonomous LLM Systems
- 2026Self-Harness: Harnesses That Improve Themselves
- 2026When Agents Implement Systems: A Case Study in Defects, Detection, and Evaluation Rigor
- 2026AgentRaft: Automated Detection of Data Over-Exposure in LLM Agents
- 2026Recognition Without Enforcement: Configuration-Dependent Failures in LLM Agent Instruction Arbitration and External Control