# Learning Deep Sketch Abstraction

arXiv: https://arxiv.org/abs/1804.04804 · HTML: https://mlsanity.com/paper/1804.04804/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Apr 2018.

## Built on by (most cited first, 30 of 36)

- [CLIPascene: Scene Sketching with Different Types and Levels of Abstraction](https://mlsanity.com/paper/2211.17256/) (2022)
- [CLIPasso: Semantically-Aware Object Sketching](https://mlsanity.com/paper/2202.05822/) (2022)
- [Breathing Life Into Sketches Using Text-to-Video Priors](https://mlsanity.com/paper/2311.13608/) (2023)
- [More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval](https://mlsanity.com/paper/2103.13990/) (2021)
- [Sketching without Worrying: Noise-Tolerant Sketch-Based Image Retrieval](https://mlsanity.com/paper/2203.14817/) (2022)
- [Differentiable Drawing and Sketching](https://mlsanity.com/paper/2103.16194/) (2021)
- [Partially Does It: Towards Scene-Level FG-SBIR with Partial Input](https://mlsanity.com/paper/2203.14804/) (2022)
- [SketchLattice: Latticed Representation for Sketch Manipulation](https://mlsanity.com/paper/2108.11636/) (2021)
- [What Can Human Sketches Do for Object Detection?](https://mlsanity.com/paper/2303.15149/) (2023)
- [Emergent Graphical Conventions in a Visual Communication Game](https://mlsanity.com/paper/2111.14210/) (2021)
- [Abstracting Sketches through Simple Primitives](https://mlsanity.com/paper/2207.13543/) (2022)
- [Exploiting Unlabelled Photos for Stronger Fine-Grained SBIR](https://mlsanity.com/paper/2303.13779/) (2023)
- [SwiftSketch: A Diffusion Model for Image-to-Vector Sketch Generation](https://mlsanity.com/paper/2502.08642/) (2025)
- [SketchINR: A First Look into Sketches as Implicit Neural Representations](https://mlsanity.com/paper/2403.09344/) (2024)
- [SketchXAI: A First Look at Explainability for Human Sketches](https://mlsanity.com/paper/2304.11744/) (2023)
- [How to Handle Sketch-Abstraction in Sketch-Based Image Retrieval?](https://mlsanity.com/paper/2403.07203/) (2024)
- [SketchGPT: Autoregressive Modeling for Sketch Generation and Recognition](https://mlsanity.com/paper/2405.03099/) (2024)
- [Deep Reinforced Attention Regression for Partial Sketch Based Image Retrieval](https://mlsanity.com/paper/2111.10917/) (2021)
- [Stroke of Surprise: Progressive Semantic Illusions in Vector Sketching](https://mlsanity.com/paper/2602.12280/) (2026)
- [Cut-and-Approximate: 3D Shape Reconstruction from Planar Cross-sections with Deep Reinforcement Learning](https://mlsanity.com/paper/2210.12509/) (2022)
- [Active Learning for Fine-Grained Sketch-Based Image Retrieval](https://mlsanity.com/paper/2309.08743/) (2023)
- [Freeview Sketching: View-Aware Fine-Grained Sketch-Based Image Retrieval](https://mlsanity.com/paper/2407.01810/) (2024)
- [Do Generalised Classifiers really work on Human Drawn Sketches?](https://mlsanity.com/paper/2407.03893/) (2024)
- [Recovering Dynamic 3D Sketches from Videos](https://mlsanity.com/paper/2503.20321/) (2025)
- [Sketch Down the FLOPs: Towards Efficient Networks for Human Sketch](https://mlsanity.com/paper/2505.23763/) (2025)
- [Multi-granularity Association Learning Framework for on-the-fly Fine-Grained Sketch-based Image Retrieval](https://mlsanity.com/paper/2201.05007/) (2022)
- [A Creative Industry Image Generation Dataset Based on Captions](https://mlsanity.com/paper/2211.09035/) (2022)
- [Conditional Human Sketch Synthesis with Explicit Abstraction Control](https://mlsanity.com/paper/2306.09274/) (2023)
- [SketchRef: a Multi-Task Evaluation Benchmark for Sketch Synthesis](https://mlsanity.com/paper/2408.08623/) (2024)
- [What is the Reward for Handwriting? -- Handwriting Generation by Imitation Learning](https://mlsanity.com/paper/2009.10962/) (2020)

## Abstract (arXiv)

Human free-hand sketches have been studied in various contexts including sketch recognition, synthesis and fine-grained sketch-based image retrieval (FG-SBIR). A fundamental challenge for sketch analysis is to deal with drastically different human drawing styles, particularly in terms of abstraction level. In this work, we propose the first stroke-level sketch abstraction model based on the insight of sketch abstraction as a process of trading off between the recognizability of a sketch and the number of strokes used to draw it. Concretely, we train a model for abstract sketch generation through reinforcement learning of a stroke removal policy that learns to predict which strokes can be safely removed without affecting recognizability. We show that our abstraction model can be used for various sketch analysis tasks including: (1) modeling stroke saliency and understanding the decision of sketch recognition models, (2) synthesizing sketches of variable abstraction for a given category, or reference object instance in a photo, and (3) training a FG-SBIR model with photos only, bypassing the expensive photo-sketch pair collection step.