Disclosure By Design: Identity Transparency as a Behavioural Property of Conversational AI Models
Not on the map. Why?
Abstract (arXiv)
As conversational AI systems become more realistic and widely deployed, users are increasingly uncertain about whether they are interacting with a human or an AI system. When AI identity is unclear, users may unwittingly share sensitive information, place unwarranted trust in AI-generated advice, or fall victim to AI-enabled fraud. More broadly, a persistent lack of transparency can erode trust in mediated communication. While regulations like the EU AI Act and California's BOT Act require AI systems to identify themselves, they provide limited guidance on reliable disclosure in real-time conversation. Existing transparency mechanisms also leave gaps: interface indicators can be omitted by deployers, and provenance tools require coordinated infrastructure and cannot provide reliable real-time verification. We ask how conversational AI systems should maintain identity transparency as human-AI interactions become more ambiguous and diverse. We advocate for disclosure by design, where AI systems explicitly disclose their artificial identity when directly asked. Implemented as model behaviour, disclosure can persist across deployment contexts without relying on user interfaces, while preserving user agency to verify identity on demand without disrupting immersive uses like role-playing. To assess current practice, we present the first multi-modal (text and voice) evaluation of disclosure behaviour in deployed systems across baseline, role-playing, and adversarial settings. We find that baseline disclosure rates are often high but drop substantially in role-play and can be suppressed under adversarial prompting. Importantly, disclosure rates vary significantly across providers and modalities, highlighting the fragility of current disclosure behaviour. We conclude with technical interventions to help developers embed disclosure as a fundamental property of conversational AI models.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 27 of its 64 references that are in the map. First posted on arXiv 27 Jan 2026.
Builds on
27 of 64 references.
References · 27
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
- 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 616
- 2022Improving alignment of dialogue agents via targeted human judgementscited by 531
- 2023Role-Play with Large Language Modelscited by 294
- 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 152
- 2022Generative Spoken Dialogue Language Modelingcited by 126
- 2021Text-Free Prosody-Aware Generative Spoken Language Modelingcited by 119
- 2024Proactive Detection of Voice Cloning with Localized Watermarkingcited by 78
- 2024Fight Back Against Jailbreaking via Prompt Adversarial Tuningcited by 67
- 2025Large Language Models Pass the Turing Testcited by 48
- 2024Pseudorandom Error-Correcting Codescited by 44
15 more
- 2025Investigating Affective Use and Emotional Well-being on ChatGPTcited by 38
- 2024SALMONN-omni: A Codec-free LLM for Full-duplex Speech Understanding and Generationcited by 33
- 2024AnthroScore: A Computational Linguistic Measure of Anthropomorphismcited by 27
- 2025How AI and Human Behaviors Shape Psychosocial Effects of Extended Chatbot Use: A Longitudinal Randomized Controlled Studycited by 22
- 2024GROOT: Generating Robust Watermark for Diffusion-Model-Based Audio Synthesiscited by 21
- 2025Neural steering vectors reveal dose and exposure-dependent impacts of human-AI relationshipscited by 21
- 2025FD-Bench: A Full-Duplex Benchmarking Pipeline Designed for Full Duplex Spoken Dialogue Systemscited by 20
- 2021The R-U-A-Robot Dataset: Helping Avoid Chatbot Deception by Detecting User Questions About Human or Non-Human Identitycited by 16
- 2024Toxicity Detection for Freecited by 15
- 2024Impact of Voice Fidelity on Decision Making: A Potential Dark Pattern?cited by 8
- 2024A Comprehensive Survey with Critical Analysis for Deepfake Speech Detectioncited by 7
- 2024Empathy Through Multimodality in Conversational Interfacescited by 6
- 2025Mapping the Parasocial AI Market: User Trends, Engagement and Riskscited by 5
- 2024Automated Adversarial Discovery for Safety Classifierscited by 3
- 2022Informing Users: Effects of Notification Properties and User Characteristics on Sharing Attitudescited by 1
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Socially fluent AI decouples conversational signals from source identity in online interaction
- 2026Dissociative Identity: Language Model Agents Lack Grounding for Reputation Mechanisms
- 2024Privacy Leakage Overshadowed by Views of AI: A Study on Human Oversight of Privacy in Language Model Agent
- 2026When Visibility Outpaces Verification: Delayed Verification and Narrative Lock-in in Agentic AI Discourse
- 2026Less Interaction But More Explanation: A Communication Perspective on Agentic AI Interfaces
- 2023"It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agents
- 2026RogueAI: A Reverse Turing Test for Detecting Licensed AI Deception in Dialogue
- 2026Lying to Win: Assessing LLM Deception through Human-AI Games and Parallel-World Probing