mlsanityData to 7 Oct 2026

SketchyCOCO: Image Generation from Freehand Scene Sketches

arXiv 2003.02683 ↗First posted 5 Mar 2020cs.CVcited by 67

Not on the map. Why?

Abstract (arXiv)

We introduce the first method for automatic image generation from scene-level freehand sketches. Our model allows for controllable image generation by specifying the synthesis goal via freehand sketches. The key contribution is an attribute vector bridged Generative Adversarial Network called EdgeGAN, which supports high visual-quality object-level image content generation without using freehand sketches as training data. We have built a large-scale composite dataset called SketchyCOCO to support and evaluate the solution. We validate our approach on the tasks of both object-level and scene-level image generation on SketchyCOCO. Through quantitative, qualitative results, human evaluation and ablation studies, we demonstrate the method's capacity to generate realistic complex scene-level images from various freehand sketches.

Key facts. Cited by 67 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 13; 2023: 10; 2024: 18; 2025: 8; 2026: 4.

67 papers.

Sort:

By area:

  1. 2022Sketch-Guided Text-to-Image Diffusion Modelscited by 180
  2. 2021Instance-Conditioned GANcited by 119
  3. 2022CLIPasso: Semantically-Aware Object Sketchingcited by 57
  4. 2021Multimodal Image Synthesis and Editing: The Generative AI Eracited by 52
  5. 2022FS-COCO: Towards Understanding of Freehand Sketches of Common Objects in Contextcited by 47
  6. 2022Modeling Image Composition for Complex Scene Generationcited by 43
  7. 2021Context-Aware Layout to Image Generation with Enhanced Object Appearancecited by 41
  8. 2021Sketch Your Own GANcited by 37
  9. 2023Picture that Sketch: Photorealistic Image Generation from Abstract Sketchescited by 34
  10. 2023ConZIC: Controllable Zero-shot Image Captioning by Sampling-Based Polishingcited by 33
  11. 2022SceneTrilogy: On Human Scene-Sketch and its Complementarity with Photo and Textcited by 30
  12. 2024SketchAgent: Language-Driven Sequential Sketch Generationcited by 30
38 more
  1. 2022A Sketch Is Worth a Thousand Words: Image Retrieval with Text and Sketchcited by 28
  2. 2021Adversarial Open Domain Adaptation for Sketch-to-Photo Synthesiscited by 24
  3. 2022DynaST: Dynamic Sparse Transformer for Exemplar-Guided Image Generationcited by 24
  4. 2024ScribbleLight: Single Image Indoor Relighting with Scribblescited by 22
  5. 2022Partially Does It: Towards Scene-Level FG-SBIR with Partial Inputcited by 20
  6. 2023What Can Human Sketches Do for Object Detection?cited by 19
  7. 2023Sketch2Saliency: Learning to Detect Salient Objects from Human Drawingscited by 17
  8. 2025A Comprehensive Survey on Composed Image Retrievalcited by 17
  9. 2022CoGS: Controllable Generation and Search from Sketch and Stylecited by 14
  10. 2023SEVA: Leveraging sketches to evaluate alignment between human and machine visual abstractioncited by 14
  11. 2025SwiftSketch: A Diffusion Model for Image-to-Vector Sketch Generationcited by 14
  12. 2021Facial-Sketch Synthesis: A New Challengecited by 13
  13. 2025Uni-Retrieval: A Multi-Style Retrieval Framework for STEM's Educationcited by 13
  14. 2024You'll Never Walk Alone: A Sketch and Text Duet for Fine-Grained Image Retrievalcited by 12
  15. 2023DiffSketching: Sketch Control Image Synthesis with Diffusion Modelscited by 10
  16. 2024Animate Your Thoughts: Decoupled Reconstruction of Dynamic Natural Vision from Slow Brain Activitycited by 10
  17. 2021One Sketch for All: One-Shot Personalized Sketch Segmentationcited by 9
  18. 2021Deep Image Synthesis from Intuitive User Input: A Review and Perspectivescited by 8
  19. 2024Text-to-Image Diffusion Models are Great Sketch-Photo Matchmakerscited by 8
  20. 2024Sketch-guided Image Inpainting with Partial Discrete Diffusion Processcited by 7
  21. 2022DeepPortraitDrawing: Generating Human Body Images from Freehand Sketchescited by 5
  22. 2023Learning Geometry-aware Representations by Sketchingcited by 5
  23. 2024Semi-supervised reference-based sketch extraction using a contrastive learning frameworkcited by 5
  24. 2024A Survey of Multimodal Composite Editing and Retrievalcited by 5
  25. 2024LineArt: A Knowledge-guided Training-free High-quality Appearance Transfer for Design Drawing with Diffusion Modelcited by 5
  26. 2022Unsupervised Scene Sketch to Photo Synthesiscited by 4
  27. 2023Open Vocabulary Semantic Scene Sketch Understandingcited by 4
  28. 2024What Sketch Explainability Really Means for Downstream Taskscited by 4
  29. 2024Sketch-Guided Scene Image Generationcited by 4
  30. 2021Data Augmentation for Object Detection via Differentiable Neural Renderingcited by 3
  31. 2021Scene Designer: a Unified Model for Scene Search and Synthesis from Sketchcited by 3
  32. 2021Sketch-based Creativity Support Tools using Deep Learningcited by 3
  33. 2023Improving Sketch Colorization using Adversarial Segmentation Consistencycited by 3
  34. 2021Adversarial Segmentation Loss for Sketch Colorizationcited by 2
  35. 2024High-Quality Medical Image Generation from Free-hand Sketchcited by 2
  36. 2024U-Sketch: An Efficient Approach for Sketch to Image Diffusion Modelscited by 2
  37. 2024Do Generalised Classifiers really work on Human Drawn Sketches?cited by 2
  38. 2024Automatic Scene Generation: State-of-the-Art Techniques, Models, Datasets, Challenges, and Future Prospectscited by 2

50 most cited of 67 All 67 (JSON)