mlsanityData to 7 Oct 2026

SketchyGAN: Towards Diverse and Realistic Sketch to Image Synthesis

arXiv 1801.02753 ↗First posted 9 Jan 2018cs.CVcited by 81

Not on the map. Why?

Abstract (arXiv)

Synthesizing realistic images from human drawn sketches is a challenging problem in computer graphics and vision. Existing approaches either need exact edge maps, or rely on retrieval of existing photographs. In this work, we propose a novel Generative Adversarial Network (GAN) approach that synthesizes plausible images from 50 categories including motorcycles, horses and couches. We demonstrate a data augmentation technique for sketches which is fully automatic, and we show that the augmented data is helpful to our task. We introduce a new network building block suitable for both the generator and discriminator which improves the information flow by injecting the input image at multiple scales. Compared to state-of-the-art image translation methods, our approach generates more realistic images and achieves significantly higher Inception Scores.

Key facts. Cited by 81 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Jan 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 19; 2023: 16; 2024: 19; 2025: 2; 2026: 3.

81 papers.

Sort:

By area:

  1. 2022Sketch-Guided Text-to-Image Diffusion Modelscited by 180
  2. 2021Image-to-Image Translation: Methods and Applicationscited by 116
  3. 2023How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMscited by 103
  4. 2023SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Fieldcited by 93
  5. 2021Multimodal Conditional Image Synthesis with Product-of-Experts GANscited by 75
  6. 2021Rebooting ACGAN: Auxiliary Classifier GANs with Stable Trainingcited by 57
  7. 2021Multimodal Image Synthesis and Editing: The Generative AI Eracited by 52
  8. 2023Control3D: Towards Controllable Text-to-3D Generationcited by 46
  9. 2022Adaptively-Realistic Image Generation from Stroke and Sketch with Diffusion Modelcited by 39
  10. 2021Sketch Your Own GANcited by 37
  11. 2025IMAGGarment: Fine-Grained Garment Generation for Controllable Fashion Designcited by 36
  12. 2023Picture that Sketch: Photorealistic Image Generation from Abstract Sketchescited by 34
38 more
  1. 2021SketchEdit: Mask-Free Local Image Manipulation with Partial Sketchescited by 32
  2. 2022Sem2NeRF: Converting Single-View Semantic Masks to Neural Radiance Fieldscited by 29
  3. 2021VTGAN: Semi-supervised Retinal Image Synthesis and Disease Prediction using Vision Transformerscited by 28
  4. 2022Sketching without Worrying: Noise-Tolerant Sketch-Based Image Retrievalcited by 27
  5. 2023MaskSketch: Unpaired Structure-guided Masked Image Generationcited by 27
  6. 2023AI-Generated Content (AIGC) for Various Data Modalities: A Surveycited by 25
  7. 2021Adversarial Open Domain Adaptation for Sketch-to-Photo Synthesiscited by 24
  8. 2024ScribbleLight: Single Image Indoor Relighting with Scribblescited by 22
  9. 2021Attack to Fool and Explain Deep Networkscited by 17
  10. 2025Science-T2I: Addressing Scientific Illusions in Image Synthesiscited by 15
  11. 2022CoGS: Controllable Generation and Search from Sketch and Stylecited by 14
  12. 2021Texture Reformer: Towards Fast and Universal Interactive Texture Transfercited by 12
  13. 2024Generative AI Models for Different Steps in Architectural Design: A Literature Reviewcited by 12
  14. 2023DiffSketching: Sketch Control Image Synthesis with Diffusion Modelscited by 10
  15. 2024How to Handle Sketch-Abstraction in Sketch-Based Image Retrieval?cited by 10
  16. 2021MOGAN: Morphologic-structure-aware Generative Learning from a Single Imagecited by 9
  17. 2021Conditional Temporal Variational AutoEncoder for Action Video Predictioncited by 9
  18. 2021Deep Image Synthesis from Intuitive User Input: A Review and Perspectivescited by 8
  19. 2024Block and Detail: Scaffolding Sketch-to-Image Generationcited by 8
  20. 2024Sketch-guided Image Inpainting with Partial Discrete Diffusion Processcited by 7
  21. 2021ReGO: Reference-Guided Outpainting for Scenery Imagecited by 6
  22. 2022Paint2Pix: Interactive Painting based Progressive Image Synthesis and Editingcited by 6
  23. 2022DeepPortraitDrawing: Generating Human Body Images from Freehand Sketchescited by 5
  24. 2024Semi-supervised reference-based sketch extraction using a contrastive learning frameworkcited by 5
  25. 2024LineArt: A Knowledge-guided Training-free High-quality Appearance Transfer for Design Drawing with Diffusion Modelcited by 5
  26. 2022Unsupervised Scene Sketch to Photo Synthesiscited by 4
  27. 2023Learning Efficient Representations for Image-Based Patent Retrievalcited by 4
  28. 2024What Sketch Explainability Really Means for Downstream Taskscited by 4
  29. 2024Sketch-Guided Scene Image Generationcited by 4
  30. 2021Scene Designer: a Unified Model for Scene Search and Synthesis from Sketchcited by 3
  31. 2021Sketch Me A Videocited by 3
  32. 2021ACNet: Approaching-and-Centralizing Network for Zero-Shot Sketch-Based Image Retrievalcited by 3
  33. 2022Human Silhouette and Skeleton Video Synthesis through Wi-Fi signalscited by 3
  34. 2023Improving Sketch Colorization using Adversarial Segmentation Consistencycited by 3
  35. 2024CustomSketching: Sketch Concept Extraction for Sketch-based Image Synthesis and Editingcited by 3
  36. 2024Locally-Focused Face Representation for Sketch-to-Image Generation Using Noise-Induced Refinementcited by 3
  37. 2021Adversarial Segmentation Loss for Sketch Colorizationcited by 2
  38. 2023Make Your Brief Stroke Real and Stereoscopic: 3D-Aware Simplified Sketch to Portrait Generationcited by 2

50 most cited of 81 All 81 (JSON)