# PlayWrite: A Multimodal System for AI Supported Narrative Co-Authoring Through Play in XR

arXiv: https://arxiv.org/abs/2603.02366 · HTML: https://mlsanity.com/paper/2603.02366/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 76 references that are in the map.
- First posted on arXiv 2 Mar 2026.

## Builds on (most cited first, 16 of 16)

- [AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts](https://mlsanity.com/paper/2110.01691/) (2021)
- [Co-Writing with Opinionated Language Models Affects Users' Views](https://mlsanity.com/paper/2302.00560/) (2023)
- [Narrative Planning: Balancing Plot and Character](https://mlsanity.com/paper/1401.3841/) (2014)
- [StoryBuddy: A Human-AI Collaborative Chatbot for Parent-Child Interactive Storytelling with Flexible Parental Involvement](https://mlsanity.com/paper/2202.06205/) (2022)
- [Choice Over Control: How Users Write with Large Language Models using Diegetic and Non-Diegetic Prompting](https://mlsanity.com/paper/2303.03199/) (2023)
- [DirectGPT: A Direct Manipulation Interface to Interact with Large Language Models](https://mlsanity.com/paper/2310.03691/) (2023)
- [Interacting with next-phrase suggestions: How suggestion systems aid and influence the cognitive processes of writing](https://mlsanity.com/paper/2208.00636/) (2022)
- [Patchview: LLM-Powered Worldbuilding with Generative Dust and Magnet Visualization](https://mlsanity.com/paper/2408.04112/) (2024)
- [StoryVerse: Towards Co-authoring Dynamic Plot with LLM-based Character Simulation via Narrative Planning](https://mlsanity.com/paper/2405.13042/) (2024)
- [CodeToon: Story Ideation, Auto Comic Generation, and Structure Mapping for Code-Driven Storytelling](https://mlsanity.com/paper/2208.12981/) (2022)
- [Visual Story-Writing: Writing by Manipulating Visual Representations of Stories](https://mlsanity.com/paper/2410.07486/) (2024)
- [TaleStream: Supporting Story Ideation with Trope Knowledge](https://mlsanity.com/paper/2309.03790/) (2023)
- [Human-AI Co-Creativity: Exploring Synergies Across Levels of Creative Collaboration](https://mlsanity.com/paper/2411.12527/) (2024)
- [Clarify: Improving Model Robustness With Natural Language Corrections](https://mlsanity.com/paper/2402.03715/) (2024)
- [WhatELSE: Shaping Narrative Spaces at Configurable Level of Abstraction for AI-bridged Interactive Storytelling](https://mlsanity.com/paper/2502.18641/) (2025)
- [Toyteller: AI-powered Visual Storytelling Through Toy-Playing with Character Symbols](https://mlsanity.com/paper/2501.13284/) (2025)

## Built on by (most cited first, 1 of 1)

- [Towards Cognitive Process-Aware Proactive Writing Support](https://mlsanity.com/paper/2608.30424/) (2026)

## Abstract (arXiv)

Current AI writing tools, which rely on text prompts, poorly support the spatial and interactive nature of storytelling where ideas emerge from direct manipulation and play. We present PlayWrite, a mixed-reality system where users author stories by directly manipulating virtual characters and props. A multi-agent AI pipeline interprets these actions into Intent Frames -structured narrative beats visualized as rearrangeable story marbles on a timeline. A large language model then transforms the user's assembled sequence into a final narrative. A user study (N=13) with writers from varying domains found that PlayWrite fosters a highly improvisational and playful process. Users treated the AI as a collaborative partner, using its unexpected responses to spark new ideas and overcome creative blocks. PlayWrite demonstrates an approach for co-creative systems that move beyond text to embrace direct manipulation and play as core interaction modalities.