# SketchyScene: Richly-Annotated Scene Sketches

arXiv: https://arxiv.org/abs/1808.02473 · HTML: https://mlsanity.com/paper/1808.02473/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Aug 2018.

## Built on by (most cited first, 30 of 32)

- [FS-COCO: Towards Understanding of Freehand Sketches of Common Objects in Context](https://mlsanity.com/paper/2203.02113/) (2022)
- [SceneTrilogy: On Human Scene-Sketch and its Complementarity with Photo and Text](https://mlsanity.com/paper/2204.11964/) (2022)
- [Position: Measure Dataset Diversity, Don't Just Claim It](https://mlsanity.com/paper/2407.08188/) (2024)
- [Adversarial Open Domain Adaptation for Sketch-to-Photo Synthesis](https://mlsanity.com/paper/2104.05703/) (2021)
- [What a MESS: Multi-Domain Evaluation of Zero-Shot Semantic Segmentation](https://mlsanity.com/paper/2306.15521/) (2023)
- [Partially Does It: Towards Scene-Level FG-SBIR with Partial Input](https://mlsanity.com/paper/2203.14804/) (2022)
- [Sketch2Saliency: Learning to Detect Salient Objects from Human Drawings](https://mlsanity.com/paper/2303.11502/) (2023)
- [Emergent Graphical Conventions in a Visual Communication Game](https://mlsanity.com/paper/2111.14210/) (2021)
- [Thinking with Visual Abstract: Enhancing Multimodal Reasoning via Visual Abstraction](https://mlsanity.com/paper/2505.20164/) (2025)
- [SEVA: Leveraging sketches to evaluate alignment between human and machine visual abstraction](https://mlsanity.com/paper/2312.03035/) (2023)
- [One Sketch for All: One-Shot Personalized Sketch Segmentation](https://mlsanity.com/paper/2112.10838/) (2021)
- [Focusing on Persons: Colorizing Old Images Learning from Modern Historical Movies](https://mlsanity.com/paper/2108.06515/) (2021)
- [Fine-Grained VR Sketching: Dataset and Insights](https://mlsanity.com/paper/2209.10008/) (2022)
- [Unsupervised Scene Sketch to Photo Synthesis](https://mlsanity.com/paper/2209.02834/) (2022)
- [Open Vocabulary Semantic Scene Sketch Understanding](https://mlsanity.com/paper/2312.12463/) (2023)
- [What Sketch Explainability Really Means for Downstream Tasks](https://mlsanity.com/paper/2403.09480/) (2024)
- [Instance Segmentation of Scene Sketches Using Natural Image Priors](https://mlsanity.com/paper/2502.09608/) (2025)
- [Sketch-based Creativity Support Tools using Deep Learning](https://mlsanity.com/paper/2111.09991/) (2021)
- [Improving Sketch Colorization using Adversarial Segmentation Consistency](https://mlsanity.com/paper/2301.08590/) (2023)
- [TreeSketchNet: From Sketch To 3D Tree Parameters Generation](https://mlsanity.com/paper/2207.12297/) (2022)
- [Class-Agnostic Visio-Temporal Scene Sketch Semantic Segmentation](https://mlsanity.com/paper/2410.00266/) (2024)
- [XCI-Sketch: Extraction of Color Information from Images for Generation of Colored Outlines and Sketches](https://mlsanity.com/paper/2108.11554/) (2021)
- [A Creative Industry Image Generation Dataset Based on Captions](https://mlsanity.com/paper/2211.09035/) (2022)
- [SketchTriplet: Self-Supervised Scenarized Sketch-Text-Image Triplet Generation](https://mlsanity.com/paper/2405.18801/) (2024)
- [Back To The Drawing Board: Rethinking Scene-Level Sketch-Based Image Retrieval](https://mlsanity.com/paper/2509.06566/) (2025)
- [Visual Intelligence through Human Interaction](https://mlsanity.com/paper/2111.06913/) (2021)
- [DrawMon: A Distributed System for Detection of Atypical Sketch Content in Concurrent Pictionary Games](https://mlsanity.com/paper/2211.05429/) (2022)
- [Text-Guided Scene Sketch-to-Photo Synthesis](https://mlsanity.com/paper/2302.06883/) (2023)
- [EnTruth: Enhancing the Traceability of Unauthorized Dataset Usage in Text-to-image Diffusion Models with Minimal and Robust Alterations](https://mlsanity.com/paper/2406.13933/) (2024)
- [Computer-aided Colorization State-of-the-science: A Survey](https://mlsanity.com/paper/2410.02288/) (2024)

## Abstract (arXiv)

We contribute the first large-scale dataset of scene sketches, SketchyScene, with the goal of advancing research on sketch understanding at both the object and scene level. The dataset is created through a novel and carefully designed crowdsourcing pipeline, enabling users to efficiently generate large quantities of realistic and diverse scene sketches. SketchyScene contains more than 29,000 scene-level sketches, 7,000+ pairs of scene templates and photos, and 11,000+ object sketches. All objects in the scene sketches have ground-truth semantic and instance masks. The dataset is also highly scalable and extensible, easily allowing augmenting and/or changing scene composition. We demonstrate the potential impact of SketchyScene by training new computational models for semantic segmentation of scene sketches and showing how the new dataset enables several applications including image retrieval, sketch colorization, editing, and captioning, etc. The dataset and code can be found at https://github.com/SketchyScene/SketchyScene.