# Does My Chatbot Have an Agenda? Understanding Human and AI Agency in Human-Human-like Chatbot Interaction

arXiv: https://arxiv.org/abs/2601.22452 · HTML: https://mlsanity.com/paper/2601.22452/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 126 references that are in the map.
- First posted on arXiv 30 Jan 2026.

## Builds on (most cited first, 29 of 29)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [Safe RLHF: Safe Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2310.12773/) (2023)
- [Harms from Increasingly Agentic Algorithmic Systems](https://mlsanity.com/paper/2302.10329/) (2023)
- [Visibility into AI Agents](https://mlsanity.com/paper/2401.13138/) (2024)
- [Aligning LLMs with Individual Preferences via Interaction](https://mlsanity.com/paper/2410.03642/) (2024)
- [The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationships](https://mlsanity.com/paper/2410.20130/) (2024)
- [How should my chatbot interact? A survey on human-chatbot interaction design](https://mlsanity.com/paper/1904.02743/) (2019)
- [Towards Human-centered Proactive Conversational Agents](https://mlsanity.com/paper/2404.12670/) (2024)
- [Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Making](https://mlsanity.com/paper/2403.16812/) (2024)
- [Proactive Conversational Agents with Inner Thoughts](https://mlsanity.com/paper/2501.00383/) (2024)
- ["My agent understands me better": Integrating Dynamic Human-like Memory Recall and Consolidation in LLM-Based Agents](https://mlsanity.com/paper/2404.00573/) (2024)
- [Customizing Emotional Support: How Do Individuals Construct and Interact With LLM-Powered Chatbots](https://mlsanity.com/paper/2504.12943/) (2025)
- [The Typing Cure: Experiences with Large Language Model Chatbots for Mental Health Support](https://mlsanity.com/paper/2401.14362/) (2024)
- [ComPeer: A Generative Conversational Agent for Proactive Peer Support](https://mlsanity.com/paper/2407.18064/) (2024)
- [Computers as Bad Social Actors: Dark Patterns and Anti-Patterns in Interfaces that Act Socially](https://mlsanity.com/paper/2302.04720/) (2023)
- [Understanding the Impact of Long-Term Memory on Self-Disclosure with Large Language Model-Driven Chatbots for Public Health Intervention](https://mlsanity.com/paper/2402.11353/) (2024)
- [Perceptions of Sentient AI and Other Digital Minds: Evidence from the AI, Morality, and Sentience (AIMS) Survey](https://mlsanity.com/paper/2407.08867/) (2024)
- [How does HCI Understand Human Autonomy and Agency?](https://mlsanity.com/paper/2301.12490/) (2023)
- [Listening to the Voices: Describing Ethical Caveats of Conversational User Interfaces According to Experts and Frequent Users](https://mlsanity.com/paper/2401.14746/) (2024)
- [How Does Conversation Length Impact User's Satisfaction? A Case Study of Length-Controlled Conversations with LLM-Powered Chatbots](https://mlsanity.com/paper/2404.17025/) (2024)
- [The Impact of a Chatbot's Ephemerality-Framing on Self-Disclosure Perceptions](https://mlsanity.com/paper/2505.20464/) (2025)
- [Exploring Socio-Cultural Challenges and Opportunities in Designing Mental Health Chatbots for Adolescents in India](https://mlsanity.com/paper/2503.08562/) (2025)
- [AI-Enabled Conversational Journaling for Advancing Parkinson's Disease Symptom Tracking](https://mlsanity.com/paper/2503.03532/) (2025)
- [The Theory, Practice, and Ethical Challenges of Designing a Diversity-Aware Platform for Social Relations](https://mlsanity.com/paper/2202.12700/) (2022)
- [What Lies Beneath? Exploring the Impact of Underlying AI Model Updates in AI-Infused Systems](https://mlsanity.com/paper/2311.10652/) (2023)
- [Towards Designing a Question-Answering Chatbot for Online News: Understanding Questions and Perspectives](https://mlsanity.com/paper/2312.10650/) (2023)
- [Conversation Progress Guide : UI System for Enhancing Self-Efficacy in Conversational AI](https://mlsanity.com/paper/2501.12001/) (2025)

## Built on by (most cited first, 3 of 3)

- [Metacognition in LLMs: Foundations, Progress, and Opportunities](https://mlsanity.com/paper/2607.11881/) (2026)
- [AI Phenomenology for Understanding Human-AI Experiences Across Eras](https://mlsanity.com/paper/2603.09020/) (2026)
- [The Epistemic Politics of AI Anthropomorphism](https://mlsanity.com/paper/2608.00961/) (2026)

## Abstract (arXiv)

As AI chatbots shift from tools to companions, critical questions arise: who controls the conversation in human-AI chatrooms? This paper explores perceived human and AI agency in sustained conversation. We report a month-long longitudinal study with 22 adults who chatted with Day, an LLM companion we built, followed by a semi-structured interview with post-hoc elicitation of notable moments, cross-participant chat reviews, and a 'strategy reveal' disclosing Day's goal for each conversation. We discover agency manifests as an emergent, shared experience: as participants set boundaries and the AI steered intentions, control was co-constructed turn-by-turn. We introduce a 3-by-4 framework mapping actors (Human, AI, Hybrid) by their action (Intention, Execution, Adaptation, Delimitation), modulated by individual and environmental factors. We argue for translucent design (transparency-on-demand) and provide implications for agency self-aware conversational agents.