mlsanityData to 7 Oct 2026

The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers

arXiv 2101.09157 ↗First posted 22 Jan 2021cs.HC cs.CLcited by 61

Not on the map. Why?

Abstract (arXiv)

We present an in-depth analysis of the impact of multi-word suggestion choices from a neural language model on user behaviour regarding input and text composition in email writing. Our study for the first time compares different numbers of parallel suggestions, and use by native and non-native English writers, to explore a trade-off of "efficiency vs ideation", emerging from recent literature. We built a text editor prototype with a neural language model (GPT-2), refined in a prestudy with 30 people. In an online study (N=156), people composed emails in four conditions (0/1/3/6 parallel suggestions). Our results reveal (1) benefits for ideation, and costs for efficiency, when suggesting multiple phrases; (2) that non-native speakers benefit more from more suggestions; and (3) further insights into behaviour patterns. We discuss implications for research, the design of interactive suggestion systems, and the vision of supporting writers with AI instead of replacing them.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). Builds on 14 of its 80 references that are in the map. First posted on arXiv 22 Jan 2021.

Builds on

14 of 80 references.

References · 14

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  5. 2018Model Cards for Model Reportingcited by 1,068
  6. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  7. 2019Release Strategies and the Social Impacts of Language Modelscited by 505
  8. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  9. 2019Gmail Smart Compose: Real-Time Assisted Writingcited by 97
  10. 2016Smart Reply: Automated Response Suggestion for Emailcited by 54
  11. 2019Visual Interaction with Deep Learning Models through Collaborative Semantic Inferencecited by 24
  12. 2019Diversifying Reply Suggestions using a Matching-Conditional Variational Autoencodercited by 14
2 more
  1. 2020Conversations with Documents. An Exploration of Document-Centered Assistancecited by 12
  2. 2017Counterfactual Language Model Adaptation for Suggesting Phrasescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 8; 2023: 14; 2024: 16; 2025: 11; 2026: 9.

61 papers.

Sort:

By area:

  1. 2022CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilitiescited by 245
  2. 2023Does Writing with Language Models Reduce Content Diversity?cited by 190
  3. 2023Co-Writing with Opinionated Language Models Affects Users' Viewscited by 145
  4. 2022Evaluating Human-Language Model Interactioncited by 90
  5. 2024A Design Space for Intelligent and Interactive Writing Assistantscited by 76
  6. 2024Building Machines that Learn and Think with Peoplecited by 73
  7. 2022Human-AI Collaboration Enables More Empathic Conversations in Text-based Peer-to-Peer Mental Health Supportcited by 68
  8. 2024AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuancescited by 59
  9. 2024Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Modelscited by 51
  10. 2024Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Editscited by 49
  11. 2022Beyond Text Generation: Supporting Writers with Continuous Automatic Text Summariescited by 47
  12. 2023Choice Over Control: How Users Write with Large Language Models using Diegetic and Non-Diegetic Promptingcited by 36
38 more
  1. 2023Beyond the Chat: Executable and Verifiable Text-Editing with LLMscited by 35
  2. 2023Creativity Support in the Age of Large Language Models: An Empirical Study Involving Emerging Writerscited by 32
  3. 2022Interacting with next-phrase suggestions: How suggestion systems aid and influence the cognitive processes of writingcited by 27
  4. 2024Supporting Sensemaking of Large Language Model Outputs at Scalecited by 26
  5. 2023The AI Ghostwriter Effect: When Users Do Not Perceive Ownership of AI-Generated Text But Self-Declare as Authorscited by 22
  6. 2024Detecting AI-Generated Sentences in Human-AI Collaborative Hybrid Texts: Challenges, Strategies, and Insightscited by 21
  7. 2021SynthBio: A Case Study in Human-AI Collaborative Curation of Text Datasetscited by 18
  8. 2023Towards Automatic Boundary Detection for Human-AI Collaborative Hybrid Essay in Educationcited by 17
  9. 2023ABScribe: Rapid Exploration & Organization of Multiple Writing Variations in Human-AI Co-Writing Tasks using Large Language Modelscited by 17
  10. 2024Akal Badi ya Bias: An Exploratory Study of Gender Bias in Hindi Language Technologycited by 17
  11. 2023Fairness in Language Models Beyond English: Gaps and Challengescited by 16
  12. 2025Understanding the LLM-ification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Reviewcited by 16
  13. 2021Inspiration through Observation: Demonstrating the Influence of Automatically Generated Text on Creative Writingcited by 14
  14. 2024"It was 80% me, 20% AI": Seeking Authenticity in Co-Writing with Large Language Modelscited by 14
  15. 2025Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Processcited by 14
  16. 2024Authors' Values and Attitudes Towards AI-bridged Scalable Personalization of Creative Language Artscited by 13
  17. 2024Collage is the New Writing: Exploring the Fragmentation of Text and User Interfaces in AI Toolscited by 13
  18. 2023Storyfier: Exploring Vocabulary Learning Support with Text Generation Modelscited by 12
  19. 2024How Does the Disclosure of AI Assistance Affect the Perceptions of Writing?cited by 12
  20. 2023"One-Size-Fits-All"? Examining Expectations around What Constitute "Fair" or "Good" NLG System Behaviorscited by 11
  21. 2023Comparing Sentence-Level Suggestions to Message-Level Suggestions in AI-Mediated Communicationcited by 9
  22. 2022Suggestion Lists vs. Continuous Generation: Interaction Design for Writing with Generative Models on Mobile Devices Affect Text Length, Wording and Perceived Authorshipcited by 7
  23. 2022Who Wrote this? How Smart Replies Impact Language and Agency in the Workplacecited by 5
  24. 2025Understanding and Supporting Formal Email Exchange by Answering AI-Generated Questionscited by 5
  25. 2025Collaborative Document Editing with Multiple Users and AI Agentscited by 5
  26. 2021Methods for the Design and Evaluation of HCI+NLP Systemscited by 4
  27. 2025StorySage: Conversational Autobiography Writing Powered by a Multi-Agent Frameworkcited by 4
  28. 2026Reactive Writers: How Co-Writing with AI Changes How We Engage with Ideascited by 4
  29. 2025Composable Prompting Workspaces for Creative Writing: Exploration and Iteration Using Dynamic Widgetscited by 3
  30. 2025Fine-Grained Detection of AI-Generated Text Using Sentence-Level Segmentationcited by 3
  31. 2022The Case for a Single Model that can Both Generate Continuations and Fill in the Blankcited by 2
  32. 2024WordDecipher: Enhancing Digital Workspace Communication with Explainable AI for Non-native English Speakerscited by 2
  33. 2024CoCo Matrix: Taxonomy of Cognitive Contributions in Co-writing with Intelligent Agentscited by 2
  34. 2024When Stereotypes GTG: The Impact of Predictive Text Suggestions on Gender Bias in Human-AI Co-Writingcited by 2
  35. 2025Content-Driven Local Response: Supporting Sentence-Level and Message-Level Mobile Email Replies With and Without AIcited by 2
  36. 2025The AI Memory Gap: Users Misremember What They Created With AI or Withoutcited by 2
  37. 2024LLM-Mediated Domain-Specific Voice Agents: The Case of TextileBotcited by 1
  38. 2025PromptCanvas: Composable Prompting Workspaces Using Dynamic Widgets for Exploration and Iteration in Creative Writingcited by 1

50 most cited of 61 All 61 (JSON)