mlsanityData to 7 Oct 2026

Interactive Neural Painting

arXiv 2307.16441 ↗First posted 31 Jul 2023cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

In the last few years, Neural Painting (NP) techniques became capable of producing extremely realistic artworks. This paper advances the state of the art in this emerging research domain by proposing the first approach for Interactive NP. Considering a setting where a user looks at a scene and tries to reproduce it on a painting, our objective is to develop a computational framework to assist the users creativity by suggesting the next strokes to paint, that can be possibly used to complete the artwork. To accomplish such a task, we propose I-Paint, a novel method based on a conditional transformer Variational AutoEncoder (VAE) architecture with a two-stage decoder. To evaluate the proposed approach and stimulate research in this area, we also introduce two novel datasets. Our experiments show that our approach provides good stroke suggestions and compares favorably to the state of the art. Additional details, code and examples are available at https://helia95.github.io/inp-website.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 38 of its 54 references that are in the map. First posted on arXiv 31 Jul 2023.

Builds on

38 of 54 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  4. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  5. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  6. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  7. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
  8. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  9. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 624
  10. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 497
  11. 2019Image2StyleGAN++: How to Edit the Embedded Images?cited by 356
  12. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
26 more
  1. 2017A Neural Representation of Sketch Drawingscited by 311
  2. 2017Toward Multimodal Image-to-Image Translationcited by 295
  3. 2019SEAN: Image Synthesis with Semantic Region-Adaptive Normalizationcited by 253
  4. 2020Semantic Photo Manipulation with a Generative Image Priorcited by 178
  5. 2021EditGAN: High-Precision Semantic Image Editingcited by 156
  6. 2016Neural Photo Editing with Introspective Adversarial Networkscited by 109
  7. 2021Paint by Wordcited by 107
  8. 2019Multi-Channel Attention Selection GAN with Cascaded Semantic Guidance for Cross-View Image Translationcited by 97
  9. 2021Diverse Image Inpainting with Bidirectional and Autoregressive Transformerscited by 80
  10. 2019Local Class-Specific and Global Image-Level Generative Adversarial Networks for Semantic-Guided Scene Generationcited by 75
  11. 2019Learning to Paint With Model-based Deep Reinforcement Learningcited by 70
  12. 2021Paint Transformer: Feed Forward Neural Painting with Stroke Predictioncited by 61
  13. 2021Anycost GANs for Interactive Image Synthesis and Editingcited by 54
  14. 2019Interactive Sketch & Fill: Multiclass Sketch-to-Image Translationcited by 51
  15. 2020Stylized Neural Paintingcited by 50
  16. 2020Sketch-BERT: Learning Sketch Bidirectional Encoder Representation from Transformers by Self-supervised Learning of Sketch Gestaltcited by 42
  17. 2021Rethinking Style Transfer: From Pixels to Parameterized Brushstrokescited by 42
  18. 2022Adaptively-Realistic Image Generation from Stroke and Sketch with Diffusion Modelcited by 39
  19. 2021Predict, Prevent, and Evaluate: Disentangled Text-Driven Image Manipulation Empowered by Pre-Trained Vision-Language Modelcited by 34
  20. 2021DeFLOCNet: Deep Image Editing via Flexible Low-level Controlscited by 26
  21. 2021Learning by Planning: Language-Guided Global Image Editingcited by 25
  22. 2020Content Masked Loss: Human-Like Brush Stroke Planning in a Reinforcement Learning Painting Agentcited by 15
  23. 2020Combining Semantic Guidance and Deep Reinforcement Learning For Generating Human Level Paintingscited by 13
  24. 2020Painting Many Pasts: Synthesizing Time Lapse Videos of Paintingscited by 7
  25. 2021Intelli-Paint: Towards Developing Human-like Painting Agentscited by 6
  26. 2022Paint2Pix: Interactive Painting based Progressive Image Synthesis and Editingcited by 6

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Loomis Painter: Reconstructing the Painting Processcited by 1