# Prototypical Human-AI Collaboration Behaviors from LLM-Assisted Writing in the Wild

arXiv: https://arxiv.org/abs/2505.16023 · HTML: https://mlsanity.com/paper/2505.16023/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 82 references that are in the map.
- First posted on arXiv 21 May 2025.

## Builds on (most cited first, 30 of 44)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [A Long Way to Go: Investigating Length Correlations in RLHF](https://mlsanity.com/paper/2310.03716/) (2023)
- [A Survey of Confidence Estimation and Calibration in Large Language Models](https://mlsanity.com/paper/2311.08298/) (2023)
- [CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities](https://mlsanity.com/paper/2201.06796/) (2022)
- [Plan-And-Write: Towards Better Automatic Storytelling](https://mlsanity.com/paper/1811.05701/) (2018)
- [Can large language models provide useful feedback on research papers? A large-scale empirical analysis](https://mlsanity.com/paper/2310.01783/) (2023)
- [Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews](https://mlsanity.com/paper/2403.07183/) (2024)
- [Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling](https://mlsanity.com/paper/2311.08718/) (2023)
- [Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversations](https://mlsanity.com/paper/2503.04761/) (2025)
- [Pearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrievers](https://mlsanity.com/paper/2311.09180/) (2023)
- [RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs](https://mlsanity.com/paper/2404.08555/) (2024)
- [Fantastic Questions and Where to Find Them: FairytaleQA -- An Authentic Dataset for Narrative Comprehension](https://mlsanity.com/paper/2203.13947/) (2022)
- [Building Machines that Learn and Think with People](https://mlsanity.com/paper/2408.03943/) (2024)
- [Evaluating Language Models for Mathematics through Interactions](https://mlsanity.com/paper/2306.01694/) (2023)
- [Aligning LLM Agents by Learning Latent Preference from User Edits](https://mlsanity.com/paper/2404.15269/) (2024)
- [Aligning LLMs with Individual Preferences via Interaction](https://mlsanity.com/paper/2410.03642/) (2024)
- [AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances](https://mlsanity.com/paper/2409.11360/) (2024)
- [Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models](https://mlsanity.com/paper/2402.11723/) (2024)
- [The Widespread Adoption of Large Language Model-Assisted Writing Across Society](https://mlsanity.com/paper/2502.09747/) (2025)
- [Analyzing and Characterizing User Intent in Information-seeking Conversations](https://mlsanity.com/paper/1804.08759/) (2018)
- [Automated Focused Feedback Generation for Scientific Writing Assistance](https://mlsanity.com/paper/2405.20477/) (2024)
- [Creativity Support in the Age of Large Language Models: An Empirical Study Involving Emerging Writers](https://mlsanity.com/paper/2309.12570/) (2023)
- [Interacting with next-phrase suggestions: How suggestion systems aid and influence the cognitive processes of writing](https://mlsanity.com/paper/2208.00636/) (2022)
- [On the Way to LLM Personalization: Learning to Remember User Conversations](https://mlsanity.com/paper/2411.13405/) (2024)
- [Beyond Summarization: Designing AI Support for Real-World Expository Writing Tasks](https://mlsanity.com/paper/2304.02623/) (2023)
- [From Text to Self: Users' Perceptions of Potential of AI on Interpersonal Communication and Self](https://mlsanity.com/paper/2310.03976/) (2023)
- [The Use of Generative Search Engines for Knowledge Work and Complex Tasks](https://mlsanity.com/paper/2404.04268/) (2024)
- [How Many Topics? Stability Analysis for Topic Models](https://mlsanity.com/paper/1404.4606/) (2014)

## Built on by (most cited first, 25 of 25)

- [Frankentext: Stitching random text fragments into long-form narratives](https://mlsanity.com/paper/2505.18128/) (2025)
- [From Words to Widgets for Controllable LLM Generation](https://mlsanity.com/paper/2604.10925/) (2026)
- [Feedback by Design: Understanding and Overcoming User Feedback Barriers in Conversational Agents](https://mlsanity.com/paper/2602.01405/) (2026)
- [Show or Tell? Modeling the evolution of request-making in Human-LLM conversations](https://mlsanity.com/paper/2508.01213/) (2025)
- [Evaluating Style-Personalized Text Generation: Challenges and Directions](https://mlsanity.com/paper/2508.06374/) (2025)
- ['Your AI Text is not Mine': Redefining and Evaluating AI-generated Text Detection under Realistic Assumptions](https://mlsanity.com/paper/2606.04906/) (2026)
- [ParaStudent: Closing the Sim2Real Gap in User Simulators for AI Tutor Evaluation](https://mlsanity.com/paper/2507.12674/) (2025)
- [The AI Memory Gap: Users Misremember What They Created With AI or Without](https://mlsanity.com/paper/2509.11851/) (2025)
- [LLUMI: Improving LLM Writing Assistance for Mental Health Support with Online Community Feedback](https://mlsanity.com/paper/2605.30273/) (2026)
- [Can You Make It Sound Like You? Post-Editing LLM-Generated Text for Personal Style](https://mlsanity.com/paper/2604.24444/) (2026)
- [Priming, Path-dependence, and Plasticity: Understanding the molding of user-LLM interaction and its implications from (many) chat logs in the wild](https://mlsanity.com/paper/2605.05767/) (2026)
- [Effects of Varying LLM Access on Essay Writing Behavior](https://mlsanity.com/paper/2606.00250/) (2026)
- [Learning from 53.6K Real-World Developer Edits of AI-Generated Code](https://mlsanity.com/paper/2607.25130/) (2026)
- [Componentization: Decomposing Monolithic LLM Responses into Manipulable Semantic Units](https://mlsanity.com/paper/2509.08203/) (2025)
- [ChatGraPhT: A Visual Conversation Interface for Multi-Path Reflection with Agentic LLM Support](https://mlsanity.com/paper/2512.22790/) (2025)
- [Tinker Tales: A Tangible Dialogue System for Child-AI Co-Creative Storytelling](https://mlsanity.com/paper/2602.04109/) (2026)
- [TUX: Measuring Human--AI Tacit Understanding](https://mlsanity.com/paper/2605.30930/) (2026)
- [Turning Intent into Specifications: A Benchmark and an Interactive User-Assistant Agent](https://mlsanity.com/paper/2606.20585/) (2026)
- [Expectation Alignment of Language Models for Real-World User Expectations](https://mlsanity.com/paper/2607.20485/) (2026)
- ["Nobody Did This": Contribution, Originality, and Accountability in Agent-Mediated Collaboration](https://mlsanity.com/paper/2607.26387/) (2026)
- [Training-Free Token-Level Steering for LLM Personalized Co-Writing](https://mlsanity.com/paper/2608.06069/) (2026)
- [When Writing Style Drifts: Benchmarking Authorship Verification under Distribution Shifts in Genre, Time and the AI-Era](https://mlsanity.com/paper/2608.17979/) (2026)
- [The Garden of Forking Prompts: How Users Explore Narrative Space in Story Generation](https://mlsanity.com/paper/2609.14677/) (2026)
- [Towards Proactive Detection of User-Side Implicit Conflicts in Human-LLM Dialogue](https://mlsanity.com/paper/2609.19155/) (2026)
- [DimSteer: Steering LLM Authoring with Automatically Discovered Stylistic Controls](https://mlsanity.com/paper/2610.04174/) (2026)

## Abstract (arXiv)

As large language models (LLMs) are used in complex writing workflows, users engage in multi-turn interactions to steer generations to better fit their needs. Rather than passively accepting output, users actively refine, explore, and co-construct text. We conduct a large-scale analysis of this collaborative behavior for users engaged in writing tasks in the wild with two popular AI assistants, Bing Copilot and WildChat. Our analysis goes beyond simple task classification or satisfaction estimation common in prior work and instead characterizes how users interact with LLMs through the course of a session. We identify prototypical behaviors in how users interact with LLMs in prompts following their original request. We refer to these as Prototypical Human-AI Collaboration Behaviors (PATHs) and find that a small group of PATHs explain a majority of the variation seen in user-LLM interaction. These PATHs span users revising intents, exploring texts, posing questions, adjusting style or injecting new content. Next, we find statistically significant correlations between specific writing intents and PATHs, revealing how users' intents shape their collaboration behaviors. We conclude by discussing the implications of our findings on LLM alignment.