# Unsupervised Doodling and Painting with Improved SPIRAL

arXiv: https://arxiv.org/abs/1910.01007 · HTML: https://mlsanity.com/paper/1910.01007/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Oct 2019.

## Built on by (most cited first, 21 of 21)

- [CLIPasso: Semantically-Aware Object Sketching](https://mlsanity.com/paper/2202.05822/) (2022)
- [Rethinking Style Transfer: From Pixels to Parameterized Brushstrokes](https://mlsanity.com/paper/2103.17185/) (2021)
- [Modern Evolution Strategies for Creativity: Fitting Concrete Images and Abstract Concepts](https://mlsanity.com/paper/2109.08857/) (2021)
- [SketchAgent: Language-Driven Sequential Sketch Generation](https://mlsanity.com/paper/2411.17673/) (2024)
- [Differentiable Drawing and Sketching](https://mlsanity.com/paper/2103.16194/) (2021)
- [SketchDreamer: Interactive Text-Augmented Creative Sketch Ideation](https://mlsanity.com/paper/2308.14191/) (2023)
- [Toward Modeling Creative Processes for Algorithmic Painting](https://mlsanity.com/paper/2205.01605/) (2022)
- [Intelli-Paint: Towards Developing Human-like Painting Agents](https://mlsanity.com/paper/2112.08930/) (2021)
- [Drawing out of Distribution with Neuro-Symbolic Generative Models](https://mlsanity.com/paper/2206.01829/) (2022)
- [Learning Geometry-aware Representations by Sketching](https://mlsanity.com/paper/2304.08204/) (2023)
- [Stroke-based Rendering: From Heuristics to Deep Learning](https://mlsanity.com/paper/2302.00595/) (2022)
- [Segmentation-Based Parametric Painting](https://mlsanity.com/paper/2311.14271/) (2023)
- [From Scratch to Sketch: Deep Decoupled Hierarchical Reinforcement Learning for Robotic Sketching Agent](https://mlsanity.com/paper/2208.04833/) (2022)
- [Cut-and-Approximate: 3D Shape Reconstruction from Planar Cross-sections with Deep Reinforcement Learning](https://mlsanity.com/paper/2210.12509/) (2022)
- [VideoSketcher: Sequential Sketch Generation Using Video Model Priors](https://mlsanity.com/paper/2602.15819/) (2026)
- [Meta-simulation for the Automated Design of Synthetic Overhead Imagery](https://mlsanity.com/paper/2209.08685/) (2022)
- [Character Generation through Self-Supervised Vectorization](https://mlsanity.com/paper/2208.02012/) (2022)
- [Deep Robot Sketching: An application of Deep Q-Learning Networks for human-like sketching](https://mlsanity.com/paper/2402.00676/) (2024)
- [Generative Visual Communication in the Era of Vision-Language Models](https://mlsanity.com/paper/2411.18727/) (2024)
- [DiffBMP: Differentiable Rendering with Bitmap Primitives](https://mlsanity.com/paper/2602.22625/) (2026)
- [Learning to See via Epiretinal Implant Stimulation in silico with Model-Based Deep Reinforcement Learning](https://mlsanity.com/paper/2606.03118/) (2026)

## Abstract (arXiv)

We investigate using reinforcement learning agents as generative models of images (extending arXiv:1804.01118). A generative agent controls a simulated painting environment, and is trained with rewards provided by a discriminator network simultaneously trained to assess the realism of the agent's samples, either unconditional or reconstructions. Compared to prior work, we make a number of improvements to the architectures of the agents and discriminators that lead to intriguing and at times surprising results. We find that when sufficiently constrained, generative agents can learn to produce images with a degree of visual abstraction, despite having only ever seen real photographs (no human brush strokes). And given enough time with the painting environment, they can produce images with considerable realism. These results show that, under the right circumstances, some aspects of human drawing can emerge from simulated embodiment, without the need for external supervision, imitation or social cues. Finally, we note the framework's potential for use in creative applications.