# Stroke-based Rendering: From Heuristics to Deep Learning

arXiv: https://arxiv.org/abs/2302.00595 · HTML: https://mlsanity.com/paper/2302.00595/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 135 references that are in the map.
- First posted on arXiv 30 Dec 2022.

## Builds on (most cited first, 30 of 45)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [A Neural Representation of Sketch Drawings](https://mlsanity.com/paper/1704.03477/) (2017)
- [VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance](https://mlsanity.com/paper/2204.08583/) (2022)
- [Neural Style Transfer: A Review](https://mlsanity.com/paper/1705.04058/) (2017)
- [CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders](https://mlsanity.com/paper/2106.14843/) (2021)
- [A guide to convolution arithmetic for deep learning](https://mlsanity.com/paper/1603.07285/) (2016)
- [Synthesizing Programs for Images using Reinforced Adversarial Learning](https://mlsanity.com/paper/1804.01118/) (2018)
- [Learning to Paint With Model-based Deep Reinforcement Learning](https://mlsanity.com/paper/1903.04411/) (2019)
- [Paint Transformer: Feed Forward Neural Painting with Stroke Prediction](https://mlsanity.com/paper/2108.03798/) (2021)
- [Deep Learning for Free-Hand Sketch: A Survey](https://mlsanity.com/paper/2001.02600/) (2020)
- [Stylized Neural Painting](https://mlsanity.com/paper/2011.08114/) (2020)
- [Learning to Sketch with Shortcut Cycle Consistency](https://mlsanity.com/paper/1805.00247/) (2018)
- [Rethinking Style Transfer: From Pixels to Parameterized Brushstrokes](https://mlsanity.com/paper/2103.17185/) (2021)
- [Modern Evolution Strategies for Creativity: Fitting Concrete Images and Abstract Concepts](https://mlsanity.com/paper/2109.08857/) (2021)
- [ColorNet: Investigating the importance of color spaces for image classification](https://mlsanity.com/paper/1902.00267/) (2019)
- [Neural Painters: A learned differentiable constraint for generating brushstroke paintings](https://mlsanity.com/paper/1904.08410/) (2019)

## Built on by (most cited first, 4 of 4)

- [Lane Graph Extraction from Aerial Imagery via Lane Segmentation Refinement with Diffusion Models](https://mlsanity.com/paper/2405.00620/) (2024)
- [Shape complexity estimation using VAE](https://mlsanity.com/paper/2304.02766/) (2023)
- [Vectorized Region Based Brush Strokes for Artistic Rendering](https://mlsanity.com/paper/2506.09969/) (2025)
- [PaintCopilot: Modeling Painting as Autonomous Artistic Continuation](https://mlsanity.com/paper/2605.20941/) (2026)

## Abstract (arXiv)

In the last few years, artistic image-making with deep learning models has gained a considerable amount of traction. A large number of these models operate directly in the pixel space and generate raster images. This is however not how most humans would produce artworks, for example, by planning a sequence of shapes and strokes to draw. Recent developments in deep learning methods help to bridge the gap between stroke-based paintings and pixel photo generation. With this survey, we aim to provide a structured introduction and understanding of common challenges and approaches in stroke-based rendering algorithms. These algorithms range from simple rule-based heuristics to stroke optimization and deep reinforcement agents, trained to paint images with differentiable vector graphics and neural rendering.