# Stylized Neural Painting

arXiv: https://arxiv.org/abs/2011.08114 · HTML: https://mlsanity.com/paper/2011.08114/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Nov 2020.

## Built on by (most cited first, 30 of 50)

- [A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?](https://mlsanity.com/paper/2303.11717/) (2023)
- [Towards Layer-wise Image Vectorization](https://mlsanity.com/paper/2206.04655/) (2022)
- [Harmonizer: Learning to Perform White-Box Image and Video Harmonization](https://mlsanity.com/paper/2207.01322/) (2022)
- [Adaptively-Realistic Image Generation from Stroke and Sketch with Diffusion Model](https://mlsanity.com/paper/2208.12675/) (2022)
- [ProcessPainter: Learn Painting Process from Sequence Data](https://mlsanity.com/paper/2406.06062/) (2024)
- [High-Fidelity Guided Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2211.17084/) (2022)
- [Customizing Text-to-Image Models with a Single Image Pair](https://mlsanity.com/paper/2405.01536/) (2024)
- [CLIPVG: Text-Guided Image Manipulation Using Differentiable Vector Graphics](https://mlsanity.com/paper/2212.02122/) (2022)
- [Grounding Visual Representations with Texts for Domain Generalization](https://mlsanity.com/paper/2207.10285/) (2022)
- [PhotoDoodle: Learning Artistic Image Editing from Few-Shot Pairwise Data](https://mlsanity.com/paper/2502.14397/) (2025)
- [Differentiable Drawing and Sketching](https://mlsanity.com/paper/2103.16194/) (2021)
- [Layered Image Vectorization via Semantic Simplification](https://mlsanity.com/paper/2406.05404/) (2024)
- [RSFNet: A White-Box Image Retouching Approach using Region-Specific Color Filters](https://mlsanity.com/paper/2303.08682/) (2023)
- [Stroke-based Neural Painting and Stylization with Dynamically Predicted Painting Region](https://mlsanity.com/paper/2309.03504/) (2023)
- [Toward Modeling Creative Processes for Algorithmic Painting](https://mlsanity.com/paper/2205.01605/) (2022)
- [Im2Oil: Stroke-Based Oil Painting Rendering with Linearly Controllable Fineness Via Adaptive Sampling](https://mlsanity.com/paper/2209.13219/) (2022)
- [Unpaired Image-to-Image Translation with Limited Data to Reveal Subtle Phenotypes](https://mlsanity.com/paper/2302.08503/) (2023)
- [VectorPainter: Advanced Stylized Vector Graphics Synthesis Using Stroke-Style Priors](https://mlsanity.com/paper/2405.02962/) (2024)
- [Intelli-Paint: Towards Developing Human-like Painting Agents](https://mlsanity.com/paper/2112.08930/) (2021)
- [Paint2Pix: Interactive Painting based Progressive Image Synthesis and Editing](https://mlsanity.com/paper/2208.08092/) (2022)
- [Inverse Painting: Reconstructing The Painting Process](https://mlsanity.com/paper/2409.20556/) (2024)
- [Learning Geometry-aware Representations by Sketching](https://mlsanity.com/paper/2304.08204/) (2023)
- [Stroke-based Rendering: From Heuristics to Deep Learning](https://mlsanity.com/paper/2302.00595/) (2022)
- [Computational Approaches for Traditional Chinese Painting: From the "Six Principles of Painting" Perspective](https://mlsanity.com/paper/2307.14227/) (2023)
- [Segmentation-Based Parametric Painting](https://mlsanity.com/paper/2311.14271/) (2023)
- [Stroke of Surprise: Progressive Semantic Illusions in Vector Sketching](https://mlsanity.com/paper/2602.12280/) (2026)
- [Convolutional Neural Network Interpretability with General Pattern Theory](https://mlsanity.com/paper/2102.04247/) (2021)
- [DeepIcon: A Hierarchical Network for Layer-wise Icon Vectorization](https://mlsanity.com/paper/2410.15760/) (2024)
- [CompArt: Operationalizing Aesthetic Alignment in Text-to-Image Generation via Principles of Art](https://mlsanity.com/paper/2503.12018/) (2025)
- [Vectorized Region Based Brush Strokes for Artistic Rendering](https://mlsanity.com/paper/2506.09969/) (2025)

## Abstract (arXiv)

This paper proposes an image-to-painting translation method that generates vivid and realistic painting artworks with controllable styles. Different from previous image-to-image translation methods that formulate the translation as pixel-wise prediction, we deal with such an artistic creation process in a vectorized environment and produce a sequence of physically meaningful stroke parameters that can be further used for rendering. Since a typical vector render is not differentiable, we design a novel neural renderer which imitates the behavior of the vector renderer and then frame the stroke prediction as a parameter searching process that maximizes the similarity between the input and the rendering output. We explored the zero-gradient problem on parameter searching and propose to solve this problem from an optimal transportation perspective. We also show that previous neural renderers have a parameter coupling problem and we re-design the rendering network with a rasterization network and a shading network that better handles the disentanglement of shape and color. Experiments show that the paintings generated by our method have a high degree of fidelity in both global appearance and local textures. Our method can be also jointly optimized with neural style transfer that further transfers visual style from other images. Our code and animated results are available at urlhttps://jiupinjia.github.io/neuralpainter/.