# Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models

arXiv: https://arxiv.org/abs/2402.11723 · HTML: https://mlsanity.com/paper/2402.11723/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 74 references that are in the map.
- First posted on arXiv 18 Feb 2024.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities](https://mlsanity.com/paper/2201.06796/) (2022)
- [Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence](https://mlsanity.com/paper/2103.08541/) (2021)
- [Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionals](https://mlsanity.com/paper/2209.14958/) (2022)
- [Co-Writing with Opinionated Language Models Affects Users' Views](https://mlsanity.com/paper/2302.00560/) (2023)
- [Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks](https://mlsanity.com/paper/2306.07899/) (2023)
- [Gmail Smart Compose: Real-Time Assisted Writing](https://mlsanity.com/paper/1906.00080/) (2019)
- [Language Models are Few-shot Multilingual Learners](https://mlsanity.com/paper/2109.07684/) (2021)
- [Sparks: Inspiration for Science Writing using Language Models](https://mlsanity.com/paper/2110.07640/) (2021)
- [The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers](https://mlsanity.com/paper/2101.09157/) (2021)
- [Smart Reply: Automated Response Suggestion for Email](https://mlsanity.com/paper/1606.04870/) (2016)
- [MERMAID: Metaphor Generation with Symbolism and Discriminative Decoding](https://mlsanity.com/paper/2103.06779/) (2021)
- [Interacting with next-phrase suggestions: How suggestion systems aid and influence the cognitive processes of writing](https://mlsanity.com/paper/2208.00636/) (2022)
- [Mapping the Design Space of Human-AI Interaction in Text Summarization](https://mlsanity.com/paper/2206.14863/) (2022)

## Built on by (most cited first, 30 of 51)

- [AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances](https://mlsanity.com/paper/2409.11360/) (2024)
- [Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Edits](https://mlsanity.com/paper/2409.14509/) (2024)
- [ChatBench: From Static Benchmarks to Human-AI Evaluation](https://mlsanity.com/paper/2504.07114/) (2025)
- [CollabStory: Multi-LLM Collaborative Story Generation and Authorship Analysis](https://mlsanity.com/paper/2406.12665/) (2024)
- [Prototypical Human-AI Collaboration Behaviors from LLM-Assisted Writing in the Wild](https://mlsanity.com/paper/2505.16023/) (2025)
- [What Makes a Good Story and How Can We Measure It? A Comprehensive Survey of Story Evaluation](https://mlsanity.com/paper/2408.14622/) (2024)
- [PersonaFlow: Designing LLM-Simulated Expert Perspectives for Enhanced Research Ideation](https://mlsanity.com/paper/2409.12538/) (2024)
- [Multi-Agents are Social Groups: Investigating Social Influence of Multiple Agents in Human-Agent Interactions](https://mlsanity.com/paper/2411.04578/) (2024)
- [Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process](https://mlsanity.com/paper/2504.12488/) (2025)
- [GPT for Games: An Updated Scoping Review (2020-2024)](https://mlsanity.com/paper/2411.00308/) (2024)
- [Can Good Writing Be Generative? Expert-Level AI Writing Emerges through Fine-Tuning on High-Quality Books](https://mlsanity.com/paper/2601.18353/) (2026)
- [Writing as a testbed for open ended agents](https://mlsanity.com/paper/2503.19711/) (2025)
- [Where Do I 'Add the Egg'?: Exploring Agency and Ownership in AI Creative Co-Writing Systems](https://mlsanity.com/paper/2509.15440/) (2025)
- [Understanding and Supporting Formal Email Exchange by Answering AI-Generated Questions](https://mlsanity.com/paper/2502.03804/) (2025)
- [DesignWeaver: Dimensional Scaffolding for Text-to-Image Product Design](https://mlsanity.com/paper/2502.09867/) (2025)
- [Modifying AI, Enhancing Essays: How Active Engagement with Generative AI Boosts Writing Quality](https://mlsanity.com/paper/2412.07200/) (2024)
- [StorySage: Conversational Autobiography Writing Powered by a Multi-Agent Framework](https://mlsanity.com/paper/2506.14159/) (2025)
- [Reactive Writers: How Co-Writing with AI Changes How We Engage with Ideas](https://mlsanity.com/paper/2603.10374/) (2026)
- [Narrix: Remixing Narrative Strategies from Examples for Story Writing](https://mlsanity.com/paper/2604.07643/) (2026)
- [MeetMap: Real-Time Collaborative Dialogue Mapping with LLMs in Online Meetings](https://mlsanity.com/paper/2502.01564/) (2025)
- [Mind the Gap! Choice Independence in Using Multilingual LLMs for Persuasive Co-Writing Tasks in Different Languages](https://mlsanity.com/paper/2502.09532/) (2025)
- [When Stereotypes GTG: The Impact of Predictive Text Suggestions on Gender Bias in Human-AI Co-Writing](https://mlsanity.com/paper/2409.20390/) (2024)
- [Scaffolding Recursive Divergence and Convergence in Story Ideation](https://mlsanity.com/paper/2507.03307/) (2025)
- [Exploration vs. Fixation: Scaffolding Divergent and Convergent Thinking for Human-AI Co-Creation with Generative Models](https://mlsanity.com/paper/2512.18388/) (2025)
- [Vidmento: Creating Video Stories Through Context-Aware Expansion With Generative Video](https://mlsanity.com/paper/2601.22013/) (2026)
- [Task-Aware Delegation Cues for LLM Agents](https://mlsanity.com/paper/2603.11011/) (2026)
- [Easy Come, Easy Go? Examining the Perceptions and Learning Effects of LLM-based Chatbot in the Context of Search-as-Learning](https://mlsanity.com/paper/2410.01396/) (2024)
- [Adaptive Human-Agent Teaming: A Review of Empirical Studies from the Process Dynamics Perspective](https://mlsanity.com/paper/2504.10918/) (2025)
- [StepWrite: Adaptive Planning for Speech-Driven Text Generation](https://mlsanity.com/paper/2508.04011/) (2025)
- [Accepted with Minor Revisions: Value of AI-Assisted Scientific Writing](https://mlsanity.com/paper/2511.12529/) (2025)

## Abstract (arXiv)

Advances in language modeling have paved the way for novel human-AI co-writing experiences. This paper explores how varying levels of scaffolding from large language models (LLMs) shape the co-writing process. Employing a within-subjects field experiment with a Latin square design, we asked participants (N=131) to respond to argumentative writing prompts under three randomly sequenced conditions: no AI assistance (control), next-sentence suggestions (low scaffolding), and next-paragraph suggestions (high scaffolding). Our findings reveal a U-shaped impact of scaffolding on writing quality and productivity (words/time). While low scaffolding did not significantly improve writing quality or productivity, high scaffolding led to significant improvements, especially benefiting non-regular writers and less tech-savvy users. No significant cognitive burden was observed while using the scaffolded writing tools, but a moderate decrease in text ownership and satisfaction was noted. Our results have broad implications for the design of AI-powered writing tools, including the need for personalized scaffolding mechanisms.