# Exploring Visual Prompts: Refining Images with Scribbles and Annotations in Generative AI Image Tools

arXiv: https://arxiv.org/abs/2503.03398 · HTML: https://mlsanity.com/paper/2503.03398/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Mar 2025.

## Built on by (most cited first, 2 of 2)

- [GenTune: Toward Traceable Prompts to Improve Controllability of Image Refinement in Environment Design](https://mlsanity.com/paper/2508.15227/) (2025)
- [Say It My Way: Exploring Control in Conversational Visual Question Answering with Blind Users](https://mlsanity.com/paper/2602.16930/) (2026)

## Abstract (arXiv)

Generative AI (GenAI) tools are increasingly integrated into design workflows. While text prompts remain the primary input method for GenAI image tools, designers often struggle to craft effective ones. Moreover, research has primarily focused on input methods for ideation, with limited attention to refinement tasks. This study explores designers' preferences for three input methods - text prompts, annotations, and scribbles - through a preliminary digital paper-based study with seven professional designers. Designers preferred annotations for spatial adjustments and referencing in-image elements, while scribbles were favored for specifying attributes such as shape, size, and position, often combined with other methods. Text prompts excelled at providing detailed descriptions or when designers sought greater GenAI creativity. However, designers expressed concerns about AI misinterpreting annotations and scribbles and the effort needed to create effective text prompts. These insights inform GenAI interface design to better support refinement tasks, align with workflows, and enhance communication with AI systems.