# Gaud'i: Conversational Interactions with Deep Representations to Generate Image Collections

arXiv: https://arxiv.org/abs/2112.04404 · HTML: https://mlsanity.com/paper/2112.04404/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 6 references that are in the map.
- First posted on arXiv 5 Dec 2021.

## Builds on (most cited first, 4 of 4)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Models](https://mlsanity.com/paper/2108.04024/) (2021)
- [BAM! The Behance Artistic Media Dataset for Recognition Beyond Photography](https://mlsanity.com/paper/1704.08614/) (2017)

## Built on by (most cited first, 2 of 2)

- [Tell Your Story: Task-Oriented Dialogs for Interactive Content Creation](https://mlsanity.com/paper/2211.03940/) (2022)
- [When Discourse Stalls: Moving Past Five Semantic Stopsigns about Generative AI in Design Research](https://mlsanity.com/paper/2503.08565/) (2025)

## Abstract (arXiv)

Based on recent advances in realistic language modeling (GPT-3) and cross-modal representations (CLIP), Gaud'i was developed to help designers search for inspirational images using natural language. In the early stages of the design process, with the goal of eliciting a client's preferred creative direction, designers will typically create thematic collections of inspirational images called "mood-boards". Creating a mood-board involves sequential image searches which are currently performed using keywords or images. Gaud'i transforms this process into a conversation where the user is gradually detailing the mood-board's theme. This representation allows our AI to generate new search queries from scratch, straight from a project briefing, following a theme hypothesized by GPT-3. Compared to previous computational approaches to mood-board creation, to the best of our knowledge, ours is the first attempt to represent mood-boards as the stories that designers tell when presenting a creative direction to a client.