# Parachute: Evaluating Interactive Human-LM Co-writing Systems

arXiv: https://arxiv.org/abs/2303.06333 · HTML: https://mlsanity.com/paper/2303.06333/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 22 references that are in the map.
- First posted on arXiv 11 Mar 2023.

## Builds on (most cited first, 9 of 9)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionals](https://mlsanity.com/paper/2209.14958/) (2022)
- [Gmail Smart Compose: Real-Time Assisted Writing](https://mlsanity.com/paper/1906.00080/) (2019)
- [Evaluating Human-Language Model Interaction](https://mlsanity.com/paper/2212.09746/) (2022)
- [Designing Creative AI Partners with COFI: A Framework for Modeling Interaction in Human-AI Co-Creative Systems](https://mlsanity.com/paper/2204.07666/) (2022)
- [STORIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generation](https://mlsanity.com/paper/2010.01717/) (2020)
- [Beyond Text Generation: Supporting Writers with Continuous Automatic Text Summaries](https://mlsanity.com/paper/2208.09323/) (2022)

## Built on by (most cited first, 7 of 7)

- [Beyond the Chat: Executable and Verifiable Text-Editing with LLMs](https://mlsanity.com/paper/2309.15337/) (2023)
- [Creativity Support in the Age of Large Language Models: An Empirical Study Involving Emerging Writers](https://mlsanity.com/paper/2309.12570/) (2023)
- [ConvXAI: Delivering Heterogeneous AI Explanations via Conversations to Support Human-AI Scientific Writing](https://mlsanity.com/paper/2305.09770/) (2023)
- [Interactive Natural Language Processing](https://mlsanity.com/paper/2305.13246/) (2023)
- [Does Human Collaboration Enhance the Accuracy of Identifying LLM-Generated Deepfake Texts?](https://mlsanity.com/paper/2304.01002/) (2023)
- [Survey of User Interface Design and Interaction Techniques in Generative AI Applications](https://mlsanity.com/paper/2410.22370/) (2024)
- [Interaction Techniques that Encourage Longer Prompts Can Improve Psychological Ownership when Writing with AI](https://mlsanity.com/paper/2507.03670/) (2025)

## Abstract (arXiv)

A surge of advances in language models (LMs) has led to significant interest in using LMs to build co-writing systems, in which humans and LMs interactively contribute to a shared writing artifact. However, there is a lack of studies assessing co-writing systems in interactive settings. We propose a human-centered evaluation framework, Parachute, for interactive co-writing systems. Parachute showcases an integrative view of interaction evaluation, where each evaluation aspect consists of categorized practical metrics. Furthermore, we present Parachute with a use case to demonstrate how to evaluate and compare co-writing systems using Parachute.