# AI-Augmented Brainwriting: Investigating the use of LLMs in group ideation

arXiv: https://arxiv.org/abs/2402.14978 · HTML: https://mlsanity.com/paper/2402.14978/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 88 references that are in the map.
- First posted on arXiv 22 Feb 2024.

## Builds on (most cited first, 10 of 10)

- [PopBlends: Strategies for Conceptual Blending with Large Language Models](https://mlsanity.com/paper/2111.04920/) (2021)
- [ChoiceMates: Supporting Unfamiliar Online Decision-Making with Multi-Agent Conversational Interactions](https://mlsanity.com/paper/2310.01331/) (2023)
- [Explainable Artificial Intelligence (XAI) from a user perspective- A synthesis of prior literature and problematizing avenues for future research](https://mlsanity.com/paper/2211.15343/) (2022)
- [Designing Participatory AI: Creative Professionals' Worries and Expectations about Generative AI](https://mlsanity.com/paper/2303.08931/) (2023)
- [Impact of Guidance and Interaction Strategies for LLM Use on Learner Performance and Perception](https://mlsanity.com/paper/2310.13712/) (2023)
- [Towards A Process Model for Co-Creating AI Experiences](https://mlsanity.com/paper/2104.07595/) (2021)
- [RECIPE: How to Integrate ChatGPT into EFL Writing Education](https://mlsanity.com/paper/2305.11583/) (2023)
- [Unreflected Acceptance -- Investigating the Negative Consequences of ChatGPT-Assisted Problem Solving in Physics Education](https://mlsanity.com/paper/2309.03087/) (2023)
- [Domain-based Latent Personal Analysis and its use for impersonation detection in social media](https://mlsanity.com/paper/2004.02346/) (2020)
- [Post-human interaction design, yes, but cautiously](https://mlsanity.com/paper/2005.05019/) (2020)

## Built on by (most cited first, 30 of 44)

- [IdeaSynth: Iterative Research Idea Development Through Evolving and Composing Idea Facets with Literature-Grounded Feedback](https://mlsanity.com/paper/2410.04025/) (2024)
- [How Far Are We From AGI: Are LLMs All We Need?](https://mlsanity.com/paper/2405.10313/) (2024)
- ['Simulacrum of Stories': Examining Large Language Models as Qualitative Research Participants](https://mlsanity.com/paper/2409.19430/) (2024)
- [Creativity in LLM-based Multi-Agent Systems: A Survey](https://mlsanity.com/paper/2505.21116/) (2025)
- [PersonaFlow: Designing LLM-Simulated Expert Perspectives for Enhanced Research Ideation](https://mlsanity.com/paper/2409.12538/) (2024)
- [Understanding the LLM-ification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Review](https://mlsanity.com/paper/2501.12557/) (2025)
- [Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process](https://mlsanity.com/paper/2504.12488/) (2025)
- [A Survey on Human-Centric LLMs](https://mlsanity.com/paper/2411.14491/) (2024)
- [Exploring the Collaborative Co-Creation Process with AI: A Case Study in Novice Music Production](https://mlsanity.com/paper/2501.15276/) (2025)
- [The Extractive-Abstractive Spectrum: Uncovering Verifiability Trade-offs in LLM Generations](https://mlsanity.com/paper/2411.17375/) (2024)
- [Language Models as Critical Thinking Tools: A Case Study of Philosophers](https://mlsanity.com/paper/2404.04516/) (2024)
- [The Great AI Witch Hunt: Reviewers Perception and (Mis)Conception of Generative AI in Research Writing](https://mlsanity.com/paper/2407.12015/) (2024)
- [AI Agent Behavioral Science](https://mlsanity.com/paper/2506.06366/) (2025)
- [Ink and Individuality: Crafting a Personalised Narrative in the Age of LLMs](https://mlsanity.com/paper/2404.00026/) (2024)
- [Co-designing Large Language Model Tools for Project-Based Learning with K12 Educators](https://mlsanity.com/paper/2502.09799/) (2025)
- [Collaborative Document Editing with Multiple Users and AI Agents](https://mlsanity.com/paper/2509.11826/) (2025)
- [Conversational Agents as Catalysts for Critical Thinking: Challenging Design Fixation in Group Design](https://mlsanity.com/paper/2406.11125/) (2024)
- [Conversational Agents as Catalysts for Critical Thinking: Challenging Social Influence in Group Decision-making](https://mlsanity.com/paper/2503.14263/) (2025)
- [Towards Pervasive Distributed Agentic Generative AI -- A State of The Art](https://mlsanity.com/paper/2506.13324/) (2025)
- [The Impact of Generative Artificial Intelligence on Ideation and the performance of Innovation Teams (Preprint)](https://mlsanity.com/paper/2410.18357/) (2024)
- [Exploring Collaborative GenAI Agents in Synchronous Group Settings: Eliciting Team Perceptions and Design Considerations for the Future of Work](https://mlsanity.com/paper/2504.14779/) (2025)
- [Human-Centered AI Communication in Co-Creativity: An Initial Framework and Insights](https://mlsanity.com/paper/2505.18385/) (2025)
- [Get on the Train or be Left on the Station: Using LLMs for Software Engineering Research](https://mlsanity.com/paper/2506.12691/) (2025)
- [Human-AI Interactions: Cognitive, Behavioral, and Emotional Impacts](https://mlsanity.com/paper/2510.17753/) (2025)
- [Guiding Generative Storytelling with Knowledge Graphs](https://mlsanity.com/paper/2505.24803/) (2025)
- [The AI Memory Gap: Users Misremember What They Created With AI or Without](https://mlsanity.com/paper/2509.11851/) (2025)
- [A Novel Mathematical Framework for Objective Characterization of Ideas](https://mlsanity.com/paper/2409.07578/) (2024)
- [Easy Come, Easy Go? Examining the Perceptions and Learning Effects of LLM-based Chatbot in the Context of Search-as-Learning](https://mlsanity.com/paper/2410.01396/) (2024)
- [Simulating Filter Bubble on Short-video Recommender System with Large Language Model Agents](https://mlsanity.com/paper/2504.08742/) (2025)
- [Adaptive Human-Agent Teaming: A Review of Empirical Studies from the Process Dynamics Perspective](https://mlsanity.com/paper/2504.10918/) (2025)

## Abstract (arXiv)

The growing availability of generative AI technologies such as large language models (LLMs) has significant implications for creative work. This paper explores twofold aspects of integrating LLMs into the creative process - the divergence stage of idea generation, and the convergence stage of evaluation and selection of ideas. We devised a collaborative group-AI Brainwriting ideation framework, which incorporated an LLM as an enhancement into the group ideation process, and evaluated the idea generation process and the resulted solution space. To assess the potential of using LLMs in the idea evaluation process, we design an evaluation engine and compared it to idea ratings assigned by three expert and six novice evaluators. Our findings suggest that integrating LLM in Brainwriting could enhance both the ideation process and its outcome. We also provide evidence that LLMs can support idea evaluation. We conclude by discussing implications for HCI education and practice.