mlsanityData to 14 Oct 2026

Interactive Drawing Guidance for Anime Illustrations with Diffusion Model

arXiv 2507.09140 ↗First posted 12 Jul 2025cs.GRcited by 0

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Creating high-quality anime illustrations presents notable challenges, particularly for beginners, due to the intricate styles and fine details inherent in anime art. We present an interactive drawing guidance system specifically designed for anime illustrations to address this issue. It offers real-time guidance to help users refine their work and streamline the creative process. Our system is built upon the StreamDiffusion pipeline to deliver real-time drawing assistance. We fine-tune Stable Diffusion with LoRA to synthesize anime style RGB images from user-provided hand-drawn sketches and prompts. Leveraging the Informative Drawings model, we transform these RGB images into rough sketches, which are further refined into structured guidance sketches using a custom-designed optimizer. The proposed system offers precise, real-time guidance aligned with the creative intent of the user, significantly enhancing both the efficiency and accuracy of the drawing process. To assess the effectiveness of our approach, we conducted a user study, gathering empirical feedback on both system performance and interface usability.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 19 of its 26 references that are in the map. First posted on arXiv 12 Jul 2025.

Builds on

19 of 26 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb

Other references · 18

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  7. 2022Classifier-Free Diffusion Guidancecited by 5,847
  8. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  9. 2017A Neural Representation of Sketch Drawingscited by 313
  10. 2023LCM-LoRA: A Universal Stable-Diffusion Acceleration Modulecited by 211
  11. 2022Learning to generate line drawings that convey geometry and semanticscited by 83
  12. 2023StreamDiffusion: A Pipeline-level Solution for Real-time Interactive Generationcited by 58
6 more
  1. 2021Sketch2Model: View-Aware 3D Modeling from Single Free-Hand Sketchescited by 54
  2. 2019Interactive Sketch & Fill: Multiclass Sketch-to-Image Translationcited by 51
  3. 2022Sketching without Worrying: Noise-Tolerant Sketch-Based Image Retrievalcited by 27
  4. 2021dualFace:Two-Stage Drawing Guidance for Freehand Portrait Sketchingcited by 8
  5. 2023AniFaceDrawing: Anime Portrait Exploration during Your Sketchingcited by 7
  6. 2024A Study on Cognitive Effects of Canvas Size for Augmenting Drawing Skillcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023AnimeDiffusion: Anime Face Line Drawing Colorization via Diffusion Models
  2. 2025AnimeColor: Reference-based Animation Colorization with Diffusion Transformers
  3. 2026AnimeAdapter: A Modular Adapter for Appearance-Consistent Anime Character Generation
  4. 2025SSIMBaD: Sigma Scaling with SSIM-Guided Balanced Diffusion for AnimeFace Colorization
  5. 2025PhysAnimator: Physics-Guided Generative Cartoon Animation
  6. 2025Image Referenced Sketch Colorization Based on Animation Creation Workflow
  7. 2024Diffutoon: High-Resolution Editable Toon Shading via Diffusion Models
  8. 2024Sketch2Manga: Shaded Manga Screening from Sketch with Diffusion Models