# Trash to Treasure: Using text-to-image models to inform the design of physical artefacts

arXiv: https://arxiv.org/abs/2302.00561 · HTML: https://mlsanity.com/paper/2302.00561/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 14 references that are in the map.
- First posted on arXiv 1 Feb 2023.

## Builds on (most cited first, 3 of 3)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [How to Prompt? Opportunities and Challenges of Zero- and Few-Shot Learning for Human-AI Interaction in Creative Applications of Generative Models](https://mlsanity.com/paper/2209.01390/) (2022)
- [When happy accidents spark creativity: Bringing collaborative speculation to life with generative AI](https://mlsanity.com/paper/2206.00533/) (2022)

## Built on by (most cited first, 3 of 3)

- [Art and the science of generative AI: A deeper dive](https://mlsanity.com/paper/2306.04141/) (2023)
- [Generative Visual Communication in the Era of Vision-Language Models](https://mlsanity.com/paper/2411.18727/) (2024)
- [TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models](https://mlsanity.com/paper/2608.15341/) (2026)

## Abstract (arXiv)

Text-to-image generative models have recently exploded in popularity and accessibility. Yet so far, use of these models in creative tasks that bridge the 2D digital world and the creation of physical artefacts has been understudied. We conduct a pilot study to investigate if and how text-to-image models can be used to assist in upstream tasks within the creative process, such as ideation and visualization, prior to a sculpture-making activity. Thirty participants selected sculpture-making materials and generated three images using the Stable Diffusion text-to-image generator, each with text prompts of their choice, with the aim of informing and then creating a physical sculpture. The majority of participants (23/30) reported that the generated images informed their sculptures, and 28/30 reported interest in using text-to-image models to help them in a creative task in the future. We identify several prompt engineering strategies and find that a participant's prompting strategy relates to their stage in the creative process. We discuss how our findings can inform support for users at different stages of the design process and for using text-to-image models for physical artefact design.