# ClearFairy: Capturing Creative Workflows through Decision Structuring, In-Situ Questioning, and Rationale Inference

arXiv: https://arxiv.org/abs/2509.14537 · HTML: https://mlsanity.com/paper/2509.14537/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 198 references that are in the map.
- First posted on arXiv 18 Sep 2025.

## Builds on (most cited first, 28 of 28)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Understanding the planning of LLM agents: A survey](https://mlsanity.com/paper/2402.02716/) (2024)
- [Magma: A Foundation Model for Multimodal AI Agents](https://mlsanity.com/paper/2502.13130/) (2025)
- [Proposer-Agent-Evaluator(PAE): Autonomous Skill Discovery For Foundation Model Internet Agents](https://mlsanity.com/paper/2412.13194/) (2024)
- [Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data](https://mlsanity.com/paper/2301.05843/) (2023)
- [Creating General User Models from Computer Use](https://mlsanity.com/paper/2505.10831/) (2025)
- [Need Help? Designing Proactive AI Assistants for Programming](https://mlsanity.com/paper/2410.04596/) (2024)
- [IdeaSynth: Iterative Research Idea Development Through Evolving and Composing Idea Facets with Literature-Grounded Feedback](https://mlsanity.com/paper/2410.04025/) (2024)
- [Cocoa: Co-Planning and Co-Execution with AI Agents](https://mlsanity.com/paper/2412.10999/) (2024)
- [The Effects of Generative AI on Design Fixation and Divergent Thinking](https://mlsanity.com/paper/2403.11164/) (2024)
- [CoQuest: Exploring Research Question Co-Creation with an LLM-based Agent](https://mlsanity.com/paper/2310.06155/) (2023)
- [Assistance or Disruption? Exploring and Evaluating the Design and Trade-offs of Proactive AI Programming Support](https://mlsanity.com/paper/2502.18658/) (2025)
- [Generating Automatic Feedback on UI Mockups with Large Language Models](https://mlsanity.com/paper/2403.13139/) (2024)
- [Enhancing Critical Thinking in Education by means of a Socratic Chatbot](https://mlsanity.com/paper/2409.05511/) (2024)
- [Supporting Self-Reflection at Scale with Large Language Models: Insights from Randomized Field Experiments in Classrooms](https://mlsanity.com/paper/2406.07571/) (2024)
- [Digi-Q: Learning Q-Value Functions for Training Device-Control Agents](https://mlsanity.com/paper/2502.15760/) (2025)
- [One vs. Many: Comprehending Accurate Information from Multiple Erroneous and Inconsistent AI Generations](https://mlsanity.com/paper/2405.05581/) (2024)
- [Auto-Intent: Automated Intent Discovery and Self-Exploration for Large Language Model Web Agents](https://mlsanity.com/paper/2410.22552/) (2024)
- [Demystifying Tacit Knowledge in Graphic Design: Characteristics, Instances, Approaches, and Guidelines](https://mlsanity.com/paper/2403.06252/) (2024)
- [AI-Instruments: Embodying Prompts as Instruments to Abstract & Reflect Graphical Interface Commands as General-Purpose Tools](https://mlsanity.com/paper/2502.18736/) (2025)
- [Soloist: Generating Mixed-Initiative Tutorials from Existing Guitar Instructional Videos Through Audio Processing](https://mlsanity.com/paper/2101.08846/) (2021)
- [TutoAI: A Cross-domain Framework for AI-assisted Mixed-media Tutorial Creation on Physical Tasks](https://mlsanity.com/paper/2403.08049/) (2024)
- [DiLogics: Creating Web Automation Programs With Diverse Logics](https://mlsanity.com/paper/2308.05828/) (2023)
- [Conversational Agents as Catalysts for Critical Thinking: Challenging Design Fixation in Group Design](https://mlsanity.com/paper/2406.11125/) (2024)
- [Understanding Collaborative Practices and Tools of Professional UX Practitioners in Software Organizations](https://mlsanity.com/paper/2302.11845/) (2023)
- [Tutorly: Turning Programming Videos Into Apprenticeship Learning Environments with LLMs](https://mlsanity.com/paper/2405.12946/) (2024)
- [Fuzzy Linkography: Automatic Graphical Summarization of Creative Activity Traces](https://mlsanity.com/paper/2502.04599/) (2025)

## Built on by (most cited first, 4 of 4)

- [CogInstrument: Modeling Cognitive Processes for Bidirectional Human-LLM Alignment in Planning Tasks](https://mlsanity.com/paper/2604.10587/) (2026)
- [Design Theater: Evaluating the Gap Between User-Facing Design Reasoning and Implementation in Generative UI Tools](https://mlsanity.com/paper/2607.22928/) (2026)
- [TailorCoPilot: Enabling Agentic Pattern Making with Version-Controlled State Tracking](https://mlsanity.com/paper/2608.25462/) (2026)
- [DocuTeam: Mixed-Initiative Multi-Agent Discussions around Evolving Documents](https://mlsanity.com/paper/2609.29309/) (2026)

## Abstract (arXiv)

Capturing professionals' decision-making in creative workflows (e.g., UI/UX) is essential for reflection, collaboration, and knowledge sharing, yet existing methods often leave rationales incomplete and implicit decisions hidden. To address this, we present the CLEAR approach, which structures reasoning into cognitive decision steps-linked units of actions, artifacts, and explanations making decisions traceable with generative AI. Building on CLEAR, we introduce ClearFairy, a think-aloud AI assistant for UI design that detects weak explanations, asks lightweight clarifying questions, and infers missing rationales. In a study with twelve professionals, 85% of ClearFairy's inferred rationales were accepted (as-is or with revisions). Notably, the system increased "strong explanations"-rationales providing sufficient causal reasoning-from 14% to 83% without adding cognitive demand. Furthermore, exploratory applications demonstrate that captured steps can enhance generative AI agents in Figma, yielding predictions better aligned with professionals and producing coherent outcomes. We release a dataset of 417 decision steps to support future research.