# Explaining CLIP through Co-Creative Drawings and Interaction

arXiv: https://arxiv.org/abs/2306.07429 · HTML: https://mlsanity.com/paper/2306.07429/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 12 references that are in the map.
- First posted on arXiv 12 Jun 2023.

## Builds on (most cited first, 4 of 4)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders](https://mlsanity.com/paper/2106.14843/) (2021)
- [Investigating Explainability of Generative AI for Code through Scenario-based Design](https://mlsanity.com/paper/2202.04903/) (2022)
- [Synthetic Books](https://mlsanity.com/paper/2201.09518/) (2022)

## Built on by (most cited first, 2 of 2)

- [Visions of Destruction: Exploring a Potential of Generative AI in Interactive Art](https://mlsanity.com/paper/2408.14644/) (2024)
- [Mining Contextualized Visual Associations from Images for Creativity Understanding](https://mlsanity.com/paper/2507.18915/) (2025)

## Abstract (arXiv)

This paper analyses a visual archive of drawings produced by an interactive robotic art installation where audience members narrated their dreams into a system powered by CLIPdraw deep learning (DL) model that interpreted and transformed their dreams into images. The resulting archive of prompt-image pairs were examined and clustered based on concept representation accuracy. As a result of the analysis, the paper proposes four groupings for describing and explaining CLIP-generated results: clear concept, text-to-text as image, indeterminacy and confusion, and lost in translation. This article offers a glimpse into a collection of dreams interpreted, mediated and given form by Artificial Intelligence (AI), showcasing oftentimes unexpected, visually compelling or, indeed, the dream-like output of the system, with the emphasis on processes and results of translations between languages, sign-systems and various modules of the installation. In the end, the paper argues that proposed clusters support better understanding of the neural model.