# Nine Potential Pitfalls when Designing Human-AI Co-Creative Systems

arXiv: https://arxiv.org/abs/2104.00358 · HTML: https://mlsanity.com/paper/2104.00358/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 33 references that are in the map.
- First posted on arXiv 1 Apr 2021.

## Builds on (most cited first, 11 of 11)

- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Understanding the Role of Individual Units in a Deep Neural Network](https://mlsanity.com/paper/2009.05041/) (2020)
- [Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview](https://mlsanity.com/paper/1912.11078/) (2019)
- [Visual Interaction with Deep Learning Models through Collaborative Semantic Inference](https://mlsanity.com/paper/1907.10739/) (2019)
- [An Interaction Framework for Studying Co-Creative AI](https://mlsanity.com/paper/1903.09709/) (2019)
- [Learning to See: You Are What You See](https://mlsanity.com/paper/2003.00902/) (2020)

## Built on by (most cited first, 12 of 12)

- [AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts](https://mlsanity.com/paper/2110.01691/) (2021)
- [Homogenization Effects of Large Language Models on Human Creative Ideation](https://mlsanity.com/paper/2402.01536/) (2024)
- [LAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editing](https://mlsanity.com/paper/2402.10294/) (2024)
- [Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI](https://mlsanity.com/paper/2302.00799/) (2023)
- [Interactive AI Alignment: Specification, Process, and Evaluation Alignment](https://mlsanity.com/paper/2311.00710/) (2023)
- [SynthBio: A Case Study in Human-AI Collaborative Curation of Text Datasets](https://mlsanity.com/paper/2111.06467/) (2021)
- [PersonaFlow: Designing LLM-Simulated Expert Perspectives for Enhanced Research Ideation](https://mlsanity.com/paper/2409.12538/) (2024)
- [fAIlureNotes: Supporting Designers in Understanding the Limits of AI Models for Computer Vision Tasks](https://mlsanity.com/paper/2302.11703/) (2023)
- [VRCopilot: Authoring 3D Layouts with Generative AI Models in VR](https://mlsanity.com/paper/2408.09382/) (2024)
- [An Ontology of Co-Creative AI Systems](https://mlsanity.com/paper/2310.07472/) (2023)
- [Exploring the Needs of Practising Musicians in Co-Creative AI Through Co-Design](https://mlsanity.com/paper/2502.09055/) (2025)
- [CHAI-DT: A Framework for Prompting Conversational Generative AI Agents to Actively Participate in Co-Creation](https://mlsanity.com/paper/2305.03852/) (2023)

## Abstract (arXiv)

This position paper examines potential pitfalls on the way towards achieving human-AI co-creation with generative models in a way that is beneficial to the users' interests. In particular, we collected a set of nine potential pitfalls, based on the literature and our own experiences as researchers working at the intersection of HCI and AI. We illustrate each pitfall with examples and suggest ideas for addressing it. Reflecting on all pitfalls, we discuss and conclude with implications for future research directions. With this collection, we hope to contribute to a critical and constructive discussion on the roles of humans and AI in co-creative interactions, with an eye on related assumptions and potential side-effects for creative practices and beyond.