# SketchyGAN: Towards Diverse and Realistic Sketch to Image Synthesis

arXiv: https://arxiv.org/abs/1801.02753 · HTML: https://mlsanity.com/paper/1801.02753/

## Key facts

- Cited by 81 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jan 2018.

## Built on by (most cited first, 30 of 81)

- [Sketch-Guided Text-to-Image Diffusion Models](https://mlsanity.com/paper/2211.13752/) (2022)
- [Image-to-Image Translation: Methods and Applications](https://mlsanity.com/paper/2101.08629/) (2021)
- [How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs](https://mlsanity.com/paper/2311.16101/) (2023)
- [SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field](https://mlsanity.com/paper/2303.13277/) (2023)
- [Multimodal Conditional Image Synthesis with Product-of-Experts GANs](https://mlsanity.com/paper/2112.05130/) (2021)
- [Rebooting ACGAN: Auxiliary Classifier GANs with Stable Training](https://mlsanity.com/paper/2111.01118/) (2021)
- [Multimodal Image Synthesis and Editing: The Generative AI Era](https://mlsanity.com/paper/2112.13592/) (2021)
- [Control3D: Towards Controllable Text-to-3D Generation](https://mlsanity.com/paper/2311.05461/) (2023)
- [Adaptively-Realistic Image Generation from Stroke and Sketch with Diffusion Model](https://mlsanity.com/paper/2208.12675/) (2022)
- [Sketch Your Own GAN](https://mlsanity.com/paper/2108.02774/) (2021)
- [IMAGGarment: Fine-Grained Garment Generation for Controllable Fashion Design](https://mlsanity.com/paper/2504.13176/) (2025)
- [Picture that Sketch: Photorealistic Image Generation from Abstract Sketches](https://mlsanity.com/paper/2303.11162/) (2023)
- [SketchEdit: Mask-Free Local Image Manipulation with Partial Sketches](https://mlsanity.com/paper/2111.15078/) (2021)
- [Sem2NeRF: Converting Single-View Semantic Masks to Neural Radiance Fields](https://mlsanity.com/paper/2203.10821/) (2022)
- [VTGAN: Semi-supervised Retinal Image Synthesis and Disease Prediction using Vision Transformers](https://mlsanity.com/paper/2104.06757/) (2021)
- [Sketching without Worrying: Noise-Tolerant Sketch-Based Image Retrieval](https://mlsanity.com/paper/2203.14817/) (2022)
- [MaskSketch: Unpaired Structure-guided Masked Image Generation](https://mlsanity.com/paper/2302.05496/) (2023)
- [AI-Generated Content (AIGC) for Various Data Modalities: A Survey](https://mlsanity.com/paper/2308.14177/) (2023)
- [Adversarial Open Domain Adaptation for Sketch-to-Photo Synthesis](https://mlsanity.com/paper/2104.05703/) (2021)
- [ScribbleLight: Single Image Indoor Relighting with Scribbles](https://mlsanity.com/paper/2411.17696/) (2024)
- [Attack to Fool and Explain Deep Networks](https://mlsanity.com/paper/2106.10606/) (2021)
- [Science-T2I: Addressing Scientific Illusions in Image Synthesis](https://mlsanity.com/paper/2504.13129/) (2025)
- [CoGS: Controllable Generation and Search from Sketch and Style](https://mlsanity.com/paper/2203.09554/) (2022)
- [Texture Reformer: Towards Fast and Universal Interactive Texture Transfer](https://mlsanity.com/paper/2112.02788/) (2021)
- [Generative AI Models for Different Steps in Architectural Design: A Literature Review](https://mlsanity.com/paper/2404.01335/) (2024)
- [DiffSketching: Sketch Control Image Synthesis with Diffusion Models](https://mlsanity.com/paper/2305.18812/) (2023)
- [How to Handle Sketch-Abstraction in Sketch-Based Image Retrieval?](https://mlsanity.com/paper/2403.07203/) (2024)
- [MOGAN: Morphologic-structure-aware Generative Learning from a Single Image](https://mlsanity.com/paper/2103.02997/) (2021)
- [Conditional Temporal Variational AutoEncoder for Action Video Prediction](https://mlsanity.com/paper/2108.05658/) (2021)
- [Block and Detail: Scaffolding Sketch-to-Image Generation](https://mlsanity.com/paper/2402.18116/) (2024)

## Abstract (arXiv)

Synthesizing realistic images from human drawn sketches is a challenging problem in computer graphics and vision. Existing approaches either need exact edge maps, or rely on retrieval of existing photographs. In this work, we propose a novel Generative Adversarial Network (GAN) approach that synthesizes plausible images from 50 categories including motorcycles, horses and couches. We demonstrate a data augmentation technique for sketches which is fully automatic, and we show that the augmented data is helpful to our task. We introduce a new network building block suitable for both the generator and discriminator which improves the information flow by injecting the input image at multiple scales. Compared to state-of-the-art image translation methods, our approach generates more realistic images and achieves significantly higher Inception Scores.