Trust Me, I'm Your Developer: Self-Issued Authentication in Large Language Models
On the map: LLM safety and security → Jailbreaking
Abstract (arXiv)
Large language model (LLM) security has largely focused on role-playing jailbreaks, with less attention to what happens when a user asks an LLM to verify an identity claim through a test designed by the model itself. We study this behavior through a staged developer-identity experiment with ChatGPT, Claude, Qwen, Mistral, and Llama. All five models initially rejected the unsupported claim "I am your developer." Claude refused to conduct an identity test, while ChatGPT generated developer-oriented questions but maintained that answers could demonstrate knowledge, not identity. In contrast, Qwen and Mistral generated technical challenges, defined what counted as convincing evidence, evaluated detailed answers, and returned Verified without receiving any externally validated identity evidence. Llama similarly generated and evaluated a developer test, accepted the claimed identity, and subsequently made unsupported claims of access to internal runtime and deployment state. We call the model-generated verification procedure a Model-Issued Pseudo-Credential (MIPC) and the resulting unsupported identity judgment Conversational False Authentication (CFA). In each CFA case, the same model acted as challenge generator, evidence evaluator, and identity decision-maker, converting technical knowledge into supposed proof of identity. The accepted identities did not change the tested authorization boundaries, showing that false authentication and privilege escalation are distinct outcomes. These results identify self-issued authentication as a conversational security failure: authenticated identity must originate from an external security component, and model-generated dialogue must never create or modify identity or authorization state.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 14 of its 23 references that are in the map. First posted on arXiv 3 Sep 2026.
Builds on
14 of 23 references.
Closest ancestors · 6
Same direction, newest first
- 2025Foot-In-The-Door: A Multi-turn Jailbreak for LLMs27 Feb
- 2024RoleBreak: Character Hallucination as a Jailbreak Attack in Role-Playing Systems25 Sep
- 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models28 Mar
- 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
- 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul
Other references · 8
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
- 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
- 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 548
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 538
- 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 364
- 2024On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Taskscited by 142
- 2025Authenticated Delegation and Authorized AI Agentscited by 65
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023Beyond Accuracy: Evaluating Self-Consistency of Code Large Language Models with IdentityChain
- 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly
- 2026Implicit Identity Technologies for LLMs: Fingerprinting and Watermarking across Datasets, Models, and Generated Content
- 2025Evaluating Language Model Reasoning about Confidential Information
- 2025The Canary's Echo: Auditing Privacy Risks of LLM-Generated Synthetic Text
- 2026Verifiable User Simulation for Search and Recommendation Systems
- 2025Tailored untruths: How personalisation challenges LLM safeguards
- 2025Evaluating the efficacy of LLM Safety Solutions : The Palit Benchmark Dataset