# Neural Painters: A learned differentiable constraint for generating brushstroke paintings

arXiv: https://arxiv.org/abs/1904.08410 · HTML: https://mlsanity.com/paper/1904.08410/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2019.

## Built on by (most cited first, 28 of 28)

- [Im2Vec: Synthesizing Vector Graphics without Vector Supervision](https://mlsanity.com/paper/2102.02798/) (2021)
- [Learning to generate line drawings that convey geometry and semantics](https://mlsanity.com/paper/2203.12691/) (2022)
- [Paint Transformer: Feed Forward Neural Painting with Stroke Prediction](https://mlsanity.com/paper/2108.03798/) (2021)
- [MakeAnything: Harnessing Diffusion Transformers for Multi-Domain Procedural Sequence Generation](https://mlsanity.com/paper/2502.01572/) (2025)
- [Rethinking Style Transfer: From Pixels to Parameterized Brushstrokes](https://mlsanity.com/paper/2103.17185/) (2021)
- [Modern Evolution Strategies for Creativity: Fitting Concrete Images and Abstract Concepts](https://mlsanity.com/paper/2109.08857/) (2021)
- [ProcessPainter: Learn Painting Process from Sequence Data](https://mlsanity.com/paper/2406.06062/) (2024)
- [Differentiable Drawing and Sketching](https://mlsanity.com/paper/2103.16194/) (2021)
- [Diffusion-Based Visual Art Creation: A Survey and New Perspectives](https://mlsanity.com/paper/2408.12128/) (2024)
- [GAN Computers Generate Arts? A Survey on Visual Arts, Music, and Literary Text Generation using Generative Adversarial Network](https://mlsanity.com/paper/2108.03857/) (2021)
- [Toward Modeling Creative Processes for Algorithmic Painting](https://mlsanity.com/paper/2205.01605/) (2022)
- [PICASSO: A Feed-Forward Framework for Parametric Inference of CAD Sketches via Rendering Self-Supervision](https://mlsanity.com/paper/2407.13394/) (2024)
- [Intelli-Paint: Towards Developing Human-like Painting Agents](https://mlsanity.com/paper/2112.08930/) (2021)
- [WISE: Whitebox Image Stylization by Example-based Learning](https://mlsanity.com/paper/2207.14606/) (2022)
- [Stroke-based Rendering: From Heuristics to Deep Learning](https://mlsanity.com/paper/2302.00595/) (2022)
- [Vector Grimoire: Codebook-based Shape Generation under Raster Image Supervision](https://mlsanity.com/paper/2410.05991/) (2024)
- [Neural 3D Strokes: Creating Stylized 3D Scenes with Vectorized 3D Strokes](https://mlsanity.com/paper/2311.15637/) (2023)
- [Collaborative Neural Painting](https://mlsanity.com/paper/2312.01800/) (2023)
- [Level of agreement between emotions generated by Artificial Intelligence and human evaluation: a methodological proposal](https://mlsanity.com/paper/2410.08332/) (2024)
- [Artificial Intelligence for Geometry-Based Feature Extraction, Analysis and Synthesis in Artistic Images: A Survey](https://mlsanity.com/paper/2412.01450/) (2024)
- [AnimatePainter: A Self-Supervised Rendering Framework for Reconstructing Painting Process](https://mlsanity.com/paper/2503.17029/) (2025)
- [Modulation Discovery with Differentiable Digital Signal Processing](https://mlsanity.com/paper/2510.06204/) (2025)
- [Stroke2Sketch: Harnessing Stroke Attributes for Training-Free Sketch Generation](https://mlsanity.com/paper/2510.16319/) (2025)
- [Birth of a Painting: Differentiable Brushstroke Reconstruction](https://mlsanity.com/paper/2511.13191/) (2025)
- [The Joy of Neural Painting](https://mlsanity.com/paper/2111.10283/) (2021)
- [Physically Embodied Deep Image Optimisation](https://mlsanity.com/paper/2201.08142/) (2022)
- [Character Generation through Self-Supervised Vectorization](https://mlsanity.com/paper/2208.02012/) (2022)
- [Data-Efficient Brushstroke Generation with Diffusion Models for Oil Painting](https://mlsanity.com/paper/2603.01103/) (2026)

## Abstract (arXiv)

We explore neural painters, a generative model for brushstrokes learned from a real non-differentiable and non-deterministic painting program. We show that when training an agent to "paint" images using brushstrokes, using a differentiable neural painter leads to much faster convergence. We propose a method for encouraging this agent to follow human-like strokes when reconstructing digits. We also explore the use of a neural painter as a differentiable image parameterization. By directly optimizing brushstrokes to activate neurons in a pre-trained convolutional network, we can directly visualize ImageNet categories and generate "ideal" paintings of each class. Finally, we present a new concept called intrinsic style transfer. By minimizing only the content loss from neural style transfer, we allow the artistic medium, in this case, brushstrokes, to naturally dictate the resulting style.