Priming, Path-dependence, and Plasticity: Understanding the molding of user-LLM interaction and its implications from (many) chat logs in the wild
Not on the map. Why?
Abstract (arXiv)
User interactions with LLMs are shaped by prior experiences and individual exploration, but in-lab studies do not provide system designers with visibility into these in-the-wild factors. This work explores a new approach to studying real-world user-LLM interactions through large-scale chat logs from the wild. Through analysis of 140K chatbot sessions from 7,955 anonymized global users over time, we demonstrate key patterns in user expressions despite varied tasks: (1) LLM users are not tabula rasa, nor are they constantly adapting; rather, interaction patterns form and stabilize rapidly through individual early trajectories; (2) Longitudinal outcomes, such as recurring text patterns and retention rates, are strongly correlated with early exploration; (3) Parallel dynamics are present, including organizing expressions by task types such as emotional support, or in response to model-version updates. These results present an ``agency paradox'': despite LLM input spaces being unconstrained and user-driven, we in fact see less user exploration. We call for design consideration surrounding the molding procedure and its incorporation in future research.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 39 of its 107 references that are in the map. First posted on arXiv 7 May 2026.
Builds on
39 of 107 references.
References · 39
- 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 461
- 2022CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilitiescited by 249
- 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 236
- 2022Grounded Copilot: How Programmers Interact with Code-Generating Modelscited by 218
- 2024WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wildcited by 218
- 2022Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionalscited by 191
- 2024Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferencescited by 144
- 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 123
- 2013A Computational Approach to Politeness with Application to Social Factorscited by 106
- 2025Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversationscited by 96
- 2023ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testingcited by 88
- 2023The Metacognitive Demands and Opportunities of Generative AIcited by 87
27 more
- 2023Understanding the Benefits and Challenges of Using Large Language Model-based Conversational Agents for Mental Well-being Supportcited by 75
- 2024Clio: Privacy-Preserving Insights into Real-World AI Usecited by 72
- 2023Evaluating Language Models for Mathematics through Interactionscited by 70
- 2023"It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agentscited by 60
- 2022Beyond Text Generation: Supporting Writers with Continuous Automatic Text Summariescited by 47
- 2024Large Language Models Can Solve Real-World Planning Rigorously with Formal Verification Toolscited by 47
- 2024AI-Augmented Brainwriting: Investigating the use of LLMs in group ideationcited by 44
- 2025Creating General User Models from Computer Usecited by 44
- 2022What is it like to program with artificial intelligence?cited by 39
- 2024A Taxonomy for Human-LLM Interaction Modes: An Initial Explorationcited by 38
- 2023Beyond the Chat: Executable and Verifiable Text-Editing with LLMscited by 35
- 2024Supporting Sensemaking of Large Language Model Outputs at Scalecited by 27
- 2025Customizing Emotional Support: How Do Individuals Construct and Interact With LLM-Powered Chatbotscited by 25
- 2025Prototypical Human-AI Collaboration Behaviors from LLM-Assisted Writing in the Wildcited by 25
- 2024Learning Agent-based Modeling with LLM Companions: Experiences of Novices and Experts Using ChatGPT & NetLogo Chatcited by 23
- 2023From Text to Self: Users' Perceptions of Potential of AI on Interpersonal Communication and Selfcited by 18
- 2024Task Supportive and Personalized Human-Large Language Model Interaction: A User Studycited by 17
- 2022Mapping the Design Space of Human-AI Interaction in Text Summarizationcited by 15
- 2025Prompting in the Dark: Assessing Human Performance in Prompt Engineering for Data Labeling When Gold Labels Are Absentcited by 15
- 2024Not Just Novelty: A Longitudinal Study on Utility and Customization of an AI Workflowcited by 14
- 2025Comparing human and LLM politeness strategies in free productioncited by 9
- 2025A Review of LLM-Assisted Ideationcited by 7
- 2025How Students (Really) Use ChatGPT: Uncovering Experiences Among Undergraduate Studentscited by 5
- 2026From Words to Widgets for Controllable LLM Generationcited by 5
- 2025Show or Tell? Modeling the evolution of request-making in Human-LLM conversationscited by 3
- 2025The Role of Human Creativity in the Presence of AI Creativity Tools at Work: A Case Study on AI-Driven Content Transformation in Journalismcited by 2
- 2023Using Logs Data to Identify When Software Engineers Experience Flow or Focused Workcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.