# A Creative Industry Image Generation Dataset Based on Captions

arXiv: https://arxiv.org/abs/2211.09035 · HTML: https://mlsanity.com/paper/2211.09035/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 51 references that are in the map.
- First posted on arXiv 16 Nov 2022.

## Builds on (most cited first, 28 of 28)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts](https://mlsanity.com/paper/2102.08981/) (2021)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors](https://mlsanity.com/paper/2203.13131/) (2022)
- [CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders](https://mlsanity.com/paper/2106.14843/) (2021)
- [SketchyGAN: Towards Diverse and Realistic Sketch to Image Synthesis](https://mlsanity.com/paper/1801.02753/) (2018)
- [Multimodal Conditional Image Synthesis with Product-of-Experts GANs](https://mlsanity.com/paper/2112.05130/) (2021)
- [Photo-Sketching: Inferring Contour Drawings from Images](https://mlsanity.com/paper/1901.00542/) (2019)
- [Artificial Intelligence in the Creative Industries: A Review](https://mlsanity.com/paper/2007.12391/) (2020)
- [Interactive Sketch & Fill: Multiclass Sketch-to-Image Translation](https://mlsanity.com/paper/1909.11081/) (2019)
- [Learning Deep Sketch Abstraction](https://mlsanity.com/paper/1804.04804/) (2018)
- [M6-UFC: Unifying Multi-Modal Controls for Conditional Image Synthesis via Non-Autoregressive Generative Transformers](https://mlsanity.com/paper/2105.14211/) (2021)
- [SketchyScene: Richly-Annotated Scene Sketches](https://mlsanity.com/paper/1808.02473/) (2018)
- [StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Translation](https://mlsanity.com/paper/2202.12362/) (2022)
- [Free-hand Sketch Synthesis with Deformable Stroke Models](https://mlsanity.com/paper/1510.02644/) (2015)
- [CIFAR-10: KNN-based Ensemble of Classifiers](https://mlsanity.com/paper/1611.04905/) (2016)
- [Synthesizing human-like sketches from natural images using a conditional convolutional decoder](https://mlsanity.com/paper/2003.07101/) (2020)
- [Sketch-to-Art: Synthesizing Stylized Art Images From Sketches](https://mlsanity.com/paper/2002.12888/) (2020)

## Built on by (most cited first, 1 of 1)

- [Survey on Controlable Image Synthesis with Deep Learning](https://mlsanity.com/paper/2307.10275/) (2023)

## Abstract (arXiv)

Most image generation methods are difficult to precisely control the properties of the generated images, such as structure, scale, shape, etc., which limits its large-scale application in creative industries such as conceptual design and graphic design, and so on. Using the prompt and the sketch is a practical solution for controllability. Existing datasets lack either prompt or sketch and are not designed for the creative industry. Here is the main contribution of our work. a) This is the first dataset that covers the 4 most important areas of creative industry domains and is labeled with prompt and sketch. b) We provide multiple reference images in the test set and fine-grained scores for each reference which are useful for measurement. c) We apply two state-of-the-art models to our dataset and then find some shortcomings, such as the prompt is more highly valued than the sketch.