Do System Prompts Leave Behavioral Fingerprints? A Large-Scale Empirical Study of Clone Detection via Output Similarity
Not on the map. Why?
Abstract (arXiv)
System prompts can be extracted from commercial LLMs with over 80% success and redeployed at zero cost, yet a prompt owner has no way to verify whether a suspected deployment is a clone. We propose Black-Box Behavioral Fingerprinting (BBF): the prompt owner registers a behavioral signature from model outputs and later tests whether a suspect deployment matches that signature more closely than an unrelated baseline. BBF requires only black-box API access. Through a large-scale study (4 model families, 8 benchmarks, 288,000 responses), we find that prompt choice explains 24.4% of output variance and same-model detection reaches AUC 0.876. Cross-model performance is bounded by detector identity, with off-diagonal AUC ranging from 0.845 (Claude as detector) down to 0.665 (Qwen) and overall mean 0.725. BBF resists non-adaptive prompt paraphrasing (AUC ≥ 0.889) and is robust to imperfect extraction, but a single-sentence formal-tone prefix can collapse detection on short structured outputs (MNLI 0.978 → 0.547), isolating style-invariant detection as the key open problem. Diagnostic Query Optimization, a zero-cost query selection rule, adds +0.120 to cross-model AUC.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). First posted on arXiv 25 Aug 2026.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Same evidence, different judgments: Evidence noncommutative in vision/speech-text conflictscited by 0
- 2026Do We Need Complex Topology Control? Distinct-Peer Random Routing Improves Cost-Efficiency in Sparse Multi-Agent Debatecited by 0
- 2026"You're Right, Let Me Fix It": How LLM Agents Damage Correct Work When Falsely Accusedcited by 0
- 2026Learning Normal Diffusion Dynamics for Backdoor Defense in Text-to-Image Modelscited by 0
- 2026Learning Normal Diffusion Dynamics for Backdoor Defense in Text-to-Image Models30 Sep 2026
- 2026"You're Right, Let Me Fix It": How LLM Agents Damage Correct Work When Falsely Accused26 Sep 2026
- 2026Do We Need Complex Topology Control? Distinct-Peer Random Routing Improves Cost-Efficiency in Sparse Multi-Agent Debate22 Sep 2026
- 2026Same evidence, different judgments: Evidence noncommutative in vision/speech-text conflicts22 Sep 2026