mlsanityData to 7 Oct 2026

Inverse Painting: Reconstructing The Painting Process

arXiv 2409.20556 ↗First posted 30 Sep 2024cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Given an input painting, we reconstruct a time-lapse video of how it may have been painted. We formulate this as an autoregressive image generation problem, in which an initially blank "canvas" is iteratively updated. The model learns from real artists by training on many painting videos. Our approach incorporates text and region understanding to define a set of painting "instructions" and updates the canvas with a novel diffusion-based renderer. The method extrapolates beyond the limited, acrylic style paintings on which it has been trained, showing plausible results for a wide range of artistic styles and genres.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 24 of its 47 references that are in the map. First posted on arXiv 30 Sep 2024.

Builds on

24 of 47 references.

References · 24

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  6. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  7. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
  8. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  9. 2017A Neural Representation of Sketch Drawingscited by 311
  10. 2018Synthesizing Programs for Images using Reinforced Adversarial Learningcited by 72
  11. 2019Learning to Paint With Model-based Deep Reinforcement Learningcited by 70
  12. 2024pix2gestalt: Amodal Segmentation by Synthesizing Wholescited by 70
12 more
  1. 2021Paint Transformer: Feed Forward Neural Painting with Stroke Predictioncited by 61
  2. 2024On the Content Bias in Fr'echet Video Distancecited by 51
  3. 2020Stylized Neural Paintingcited by 50
  4. 2022DALLE-2 is Seeing Double: Flaws in Word-to-Concept Mapping in Text2Image Modelscited by 50
  5. 2021Rethinking Style Transfer: From Pixels to Parameterized Brushstrokescited by 42
  6. 2023Stroke-based Neural Painting and Stylization with Dynamically Predicted Painting Regioncited by 14
  7. 2020Combining Semantic Guidance and Deep Reinforcement Learning For Generating Human Level Paintingscited by 13
  8. 2020Painting Many Pasts: Synthesizing Time Lapse Videos of Paintingscited by 7
  9. 2021Intelli-Paint: Towards Developing Human-like Painting Agentscited by 6
  10. 2018Unsupervised Image to Sequence Translation with Canvas-Drawer Networkscited by 5
  11. 2023Total Selfie: Generating Full-Body Selfiescited by 5
  12. 2019PaintBot: A Reinforcement Learning Approach for Natural Media Paintingcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 1.

6 papers.

  1. 2025Loom: Diffusion-Transformer for Interleaved Generationcited by 3
  2. 2025AnimatePainter: A Self-Supervised Rendering Framework for Reconstructing Painting Processcited by 1
  3. 2025PPJudge: Towards Human-Aligned Assessment of Artistic Painting Processcited by 1
  4. 2025Birth of a Painting: Differentiable Brushstroke Reconstructioncited by 1
  5. 2025Loomis Painter: Reconstructing the Painting Processcited by 1
  6. 2026PaintCopilot: Modeling Painting as Autonomous Artistic Continuationcited by 0