# Learning to Sketch with Deep Q Networks and Demonstrated Strokes

arXiv: https://arxiv.org/abs/1810.05977 · HTML: https://mlsanity.com/paper/1810.05977/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Oct 2018.

## Built on by (most cited first, 6 of 6)

- [MakeAnything: Harnessing Diffusion Transformers for Multi-Domain Procedural Sequence Generation](https://mlsanity.com/paper/2502.01572/) (2025)
- [DoodleFormer: Creative Sketch Drawing with Transformers](https://mlsanity.com/paper/2112.03258/) (2021)
- [Cut-and-Approximate: 3D Shape Reconstruction from Planar Cross-sections with Deep Reinforcement Learning](https://mlsanity.com/paper/2210.12509/) (2022)
- [Generative Adversarial Networks with Conditional Neural Movement Primitives for An Interactive Generative Drawing Tool](https://mlsanity.com/paper/2111.14934/) (2021)
- [Character Generation through Self-Supervised Vectorization](https://mlsanity.com/paper/2208.02012/) (2022)
- [Generative Visual Communication in the Era of Vision-Language Models](https://mlsanity.com/paper/2411.18727/) (2024)

## Abstract (arXiv)

Doodling is a useful and common intelligent skill that people can learn and master. In this work, we propose a two-stage learning framework to teach a machine to doodle in a simulated painting environment via Stroke Demonstration and deep Q-learning (SDQ). The developed system, Doodle-SDQ, generates a sequence of pen actions to reproduce a reference drawing and mimics the behavior of human painters. In the first stage, it learns to draw simple strokes by imitating in supervised fashion from a set of strokeaction pairs collected from artist paintings. In the second stage, it is challenged to draw real and more complex doodles without ground truth actions; thus, it is trained with Qlearning. Our experiments confirm that (1) doodling can be learned without direct stepby- step action supervision and (2) pretraining with stroke demonstration via supervised learning is important to improve performance. We further show that Doodle-SDQ is effective at producing plausible drawings in different media types, including sketch and watercolor.