mlsanityData to 7 Oct 2026

Sketch Your Own GAN

arXiv 2108.02774 ↗First posted 5 Aug 2021cs.CV cs.LGcited by 37

Not on the map. Why?

Abstract (arXiv)

Can a user create a deep generative model by sketching a single example? Traditionally, creating a GAN model has required the collection of a large-scale dataset of exemplars and specialized knowledge in deep learning. In contrast, sketching is possibly the most universally accessible way to convey a visual concept. In this work, we present a method, GAN Sketching, for rewriting GANs with one or more sketches, to make GANs training easier for novice users. In particular, we change the weights of an original GAN model according to user sketches. We encourage the model's output to match the user sketches through a cross-domain adversarial loss. Furthermore, we explore different regularization methods to preserve the original model's diversity and image quality. Experiments have shown that our method can mold GANs to match shapes and poses specified by sketches while maintaining realism and diversity. Finally, we demonstrate a few applications of the resulting GAN, including latent space interpolation and image editing.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 55 of its 76 references that are in the map. First posted on arXiv 5 Aug 2021.

Builds on

55 of 76 references.

References · 55

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  4. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  6. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  7. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  8. 2016Improved Techniques for Training GANscited by 2,938
  9. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  10. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  11. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  12. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
38 more
  1. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  2. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  3. 2017Adversarial Discriminative Domain Adaptationcited by 1,309
  4. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  5. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  6. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  7. 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,079
  8. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  9. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  10. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  11. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  12. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  13. 2018Which Training Methods for GANs do actually Converge?cited by 672
  14. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
  15. 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 422
  16. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  17. 2020Differentiable Augmentation for Data-Efficient GAN Trainingcited by 361
  18. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
  19. 2017A Neural Representation of Sketch Drawingscited by 311
  20. 2016Learning from Simulated and Unsupervised Images through Adversarial Trainingcited by 287
  21. 2019DRIT++: Diverse Image-to-Image Translation via Disentangled Representationscited by 244
  22. 2019Few-Shot Unsupervised Image-to-Image Translationcited by 222
  23. 2020Semantic Photo Manipulation with a Generative Image Priorcited by 178
  24. 2018Transferring GANs: generating images from limited datacited by 134
  25. 2020Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANscited by 132
  26. 2020Few-shot Image Generation with Elastic Weight Consolidationcited by 122
  27. 2019MineGAN: effective knowledge transfer from GANs to target domains with few imagescited by 106
  28. 2020Rewriting a Deep Generative Modelcited by 105
  29. 2019Image Generation From Small Datasets via Batch Statistics Adaptationcited by 102
  30. 2020RL-CycleGAN: Reinforcement Learning Aware Simulation-To-Realcited by 89
  31. 2018SketchyGAN: Towards Diverse and Realistic Sketch to Image Synthesiscited by 81
  32. 2020Sketchformer: Transformer-based Representation for Sketched Structurecited by 78
  33. 2020Image Augmentations for GAN Trainingcited by 78
  34. 2016Scribbler: Controlling Deep Image Synthesis with Sketch and Colorcited by 77
  35. 2020Transforming and Projecting Images into Class-conditional Generative Networkscited by 76
  36. 2017Deep Sketch Hashing: Fast Free-hand Sketch-Based Image Retrievalcited by 68
  37. 2019Photo-Sketching: Inferring Contour Drawings from Imagescited by 67
  38. 2020SketchyCOCO: Image Generation from Freehand Scene Sketchescited by 67

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 7; 2023: 13; 2024: 7; 2025: 3; 2026: 2.

37 papers.

Sort:

By area:

  1. 2023Erasing Concepts from Diffusion Modelscited by 530
  2. 2023Ablating Concepts in Text-to-Image Diffusion Modelscited by 312
  3. 2021CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directionscited by 78
  4. 2022FS-COCO: Towards Understanding of Freehand Sketches of Common Objects in Contextcited by 47
  5. 2022Adaptively-Realistic Image Generation from Stroke and Sketch with Diffusion Modelcited by 39
  6. 2024Customizing Text-to-Image Models with a Single Image Paircited by 36
  7. 2022State-of-the-Art in the Architecture, Methods and Applications of StyleGANcited by 34
  8. 2023Picture that Sketch: Photorealistic Image Generation from Abstract Sketchescited by 34
  9. 2021SketchEdit: Mask-Free Local Image Manipulation with Partial Sketchescited by 32
  10. 2022Sketching without Worrying: Noise-Tolerant Sketch-Based Image Retrievalcited by 27
  11. 2023MaLP: Manipulation Localization Using a Proactive Schemecited by 26
  12. 2022Rewriting Geometric Rules of a GANcited by 25
25 more
  1. 2021Facial-Sketch Synthesis: A New Challengecited by 13
  2. 2023Domain Expansion of Image Generatorscited by 13
  3. 2023DiffSketching: Sketch Control Image Synthesis with Diffusion Modelscited by 10
  4. 2024How to Handle Sketch-Abstraction in Sketch-Based Image Retrieval?cited by 10
  5. 2023MyStyle++: A Controllable Personalized Generative Priorcited by 8
  6. 2021DyStyle: Dynamic Neural Network for Multi-Attribute-Conditioned Style Editingcited by 7
  7. 2022Content-Based Search for Deep Generative Modelscited by 7
  8. 2022Extracting Semantic Knowledge from GANs with Unsupervised Learningcited by 5
  9. 2023Learning Geometry-aware Representations by Sketchingcited by 5
  10. 2024LineArt: A Knowledge-guided Training-free High-quality Appearance Transfer for Design Drawing with Diffusion Modelcited by 5
  11. 2024What Sketch Explainability Really Means for Downstream Taskscited by 4
  12. 2021ACNet: Approaching-and-Centralizing Network for Zero-Shot Sketch-Based Image Retrievalcited by 3
  13. 2023Inversion-by-Inversion: Exemplar-based Sketch-to-Photo Synthesis via Stochastic Differential Equations without Trainingcited by 2
  14. 2024U-Sketch: An Efficient Approach for Sketch to Image Diffusion Modelscited by 2
  15. 2024Do Generalised Classifiers really work on Human Drawn Sketches?cited by 2
  16. 2023Intriguing Property and Counterfactual Explanation of GAN for Remote Sensing Image Generationcited by 1
  17. 2026Controlling Your Image via Simplified Vector Graphicscited by 1
  18. 2026SketchingReality: From Freehand Scene Sketches To Photorealistic Imagescited by 1
  19. 2023Texturize a GAN Using a Single Imagecited by 0
  20. 2023Sketch-based Video Object Localizationcited by 0
  21. 2023Customize StyleGAN with One Hand Sketchcited by 0
  22. 2024SketchDeco: Training-Free Latent Composition for Precise Sketch Colourisationcited by 0
  23. 2025DiffS-NOCS: 3D Point Cloud Reconstruction through Coloring Sketches to NOCS Maps Using Diffusion Modelscited by 0
  24. 2025Annotation-Free Human Sketch Quality Assessmentcited by 0
  25. 2025MSN: Multi-directional Similarity Network for Hand-crafted and Deep-synthesized Copy-Move Forgery Detectioncited by 0