# PaintBot: A Reinforcement Learning Approach for Natural Media Painting

arXiv: https://arxiv.org/abs/1904.02201 · HTML: https://mlsanity.com/paper/1904.02201/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Apr 2019.

## Built on by (most cited first, 4 of 4)

- [Rethinking Style Transfer: From Pixels to Parameterized Brushstrokes](https://mlsanity.com/paper/2103.17185/) (2021)
- [Inverse Painting: Reconstructing The Painting Process](https://mlsanity.com/paper/2409.20556/) (2024)
- [Stroke-based Rendering: From Heuristics to Deep Learning](https://mlsanity.com/paper/2302.00595/) (2022)
- [DiffBMP: Differentiable Rendering with Bitmap Primitives](https://mlsanity.com/paper/2602.22625/) (2026)

## Abstract (arXiv)

We propose a new automated digital painting framework, based on a painting agent trained through reinforcement learning. To synthesize an image, the agent selects a sequence of continuous-valued actions representing primitive painting strokes, which are accumulated on a digital canvas. Action selection is guided by a given reference image, which the agent attempts to replicate subject to the limitations of the action space and the agent's learned policy. The painting agent policy is determined using a variant of proximal policy optimization reinforcement learning. During training, our agent is presented with patches sampled from an ensemble of reference images. To accelerate training convergence, we adopt a curriculum learning strategy, whereby reference patches are sampled according to how challenging they are using the current policy. We experiment with differing loss functions, including pixel-wise and perceptual loss, which have consequent differing effects on the learned policy. We demonstrate that our painting agent can learn an effective policy with a high dimensional continuous action space comprising pen pressure, width, tilt, and color, for a variety of painting styles. Through a coarse-to-fine refinement process our agent can paint arbitrarily complex images in the desired style.