mlsanityData to 14 Oct 2026

DiffSketcher: Text Guided Vector Sketch Synthesis through Latent Diffusion Models

arXiv 2306.14685 ↗First posted 26 Jun 2023cs.CV cs.AIcited by 54

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

We demonstrate that pre-trained text-to-image diffusion models, despite being trained on raster images, possess a remarkable capacity to guide vector sketch synthesis. In this paper, we introduce DiffSketcher, a novel algorithm for generating vectorized free-hand sketches directly from natural language prompts. Our method optimizes a set of B'ezier curves via an extended Score Distillation Sampling (SDS) loss, successfully bridging a raster-level diffusion prior with a parametric vector generator. To further accelerate the generation process, we propose a stroke initialization strategy driven by the diffusion model's intrinsic attention maps. Results show that DiffSketcher produces sketches across varying levels of abstraction while maintaining the structural integrity and essential visual details of the subject. Experiments confirm that our approach yields superior perceptual quality and controllability over existing methods. The code and demo are available at https://ximinng.github.io/DiffSketcher-project/

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 39 of its 47 references that are in the map. First posted on arXiv 26 Jun 2023.

Builds on

39 of 47 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models21 Nov
  2. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  6. 2020Denoising Diffusion Implicit Modelscited by 9,781
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  10. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  11. 2022Classifier-Free Diffusion Guidancecited by 5,847
  12. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
23 more
  1. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  2. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  3. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  4. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  5. 2015Holistically-Nested Edge Detectioncited by 592
  6. 2017A Neural Representation of Sketch Drawingscited by 313
  7. 2021CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoderscited by 187
  8. 2021Im2Vec: Synthesizing Vector Graphics without Vector Supervisioncited by 116
  9. 2022Learning to generate line drawings that convey geometry and semanticscited by 83
  10. 2022Towards Layer-wise Image Vectorizationcited by 79
  11. 2019Photo-Sketching: Inferring Contour Drawings from Imagescited by 68
  12. 2022CLIPascene: Scene Sketching with Different Types and Levels of Abstractioncited by 64
  13. 2020Creative Sketch Generationcited by 44
  14. 2021Rethinking Style Transfer: From Pixels to Parameterized Brushstrokescited by 42
  15. 2021Modern Evolution Strategies for Creativity: Fitting Concrete Images and Abstract Conceptscited by 40
  16. 2019Unsupervised Sketch-to-Photo Synthesiscited by 38
  17. 2021ClipGen: A Deep Generative Model for Clipart Vectorization and Synthesiscited by 23
  18. 2021SketchLattice: Latticed Representation for Sketch Manipulationcited by 19
  19. 2021DoodleFormer: Creative Sketch Drawing with Transformerscited by 17
  20. 2015Free-hand Sketch Synthesis with Deformable Stroke Modelscited by 16
  21. 2021StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Synthesiscited by 15
  22. 2022CLIP-CLOP: CLIP-Guided Collage and Photomontagecited by 11
  23. 2020Sketch Generation with Drawing Process Guided by Vector Flow and Grayscalecited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 14; 2025: 20; 2026: 15.

54 papers.

Sort:

By area:

  1. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  2. 2023Diffusion Model is Secretly a Training-free Open Vocabulary Semantic Segmentercited by 76
  3. 2023SVGDreamer: Text Guided SVG Generation with Diffusion Modelcited by 76
  4. 2024Empowering LLMs to Understand and Generate Complex Vector Graphicscited by 66
  5. 2023Posterior Distillation Samplingcited by 46
  6. 2023COLE: A Hierarchical Generation Framework for Multi-Layered and Editable Graphic Designcited by 43
  7. 2024Rethinking Score Distillation as a Bridge Between Image Distributionscited by 38
  8. 2024Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Modelscited by 38
  9. 2024SketchAgent: Language-Driven Sequential Sketch Generationcited by 31
  10. 2025NeuralSVG: An Implicit Representation for Text-to-Vector Generationcited by 17
  11. 2024Whiteboard-of-Thought: Thinking Step-by-Step Across Modalitiescited by 16
  12. 2025SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generationcited by 16
38 more
  1. 2025SVGen: Interpretable Vector Graphics Generation with Large Language Modelscited by 14
  2. 2024SketchINR: A First Look into Sketches as Implicit Neural Representationscited by 13
  3. 2025U-StyDiT: Ultra-high Quality Artistic Style Transfer Using Diffusion Transformerscited by 12
  4. 2026Render-in-the-Loop: Vector Graphics Generation via Visual Self-Feedbackcited by 12
  5. 2024Diff3DS: Generating View-Consistent 3D Sketch via Differentiable Curve Renderingcited by 9
  6. 2025OracleFusion: Assisting the Decipherment of Oracle Bone Script with Structurally Constrained Semantic Typographycited by 8
  7. 2025DuetSVG: Unified Multimodal SVG Generation with Internal Visual Guidancecited by 8
  8. 2024VectorPainter: Advanced Stylized Vector Graphics Synthesis Using Stroke-Style Priorscited by 7
  9. 2024Enhancing Sketch Animation: Text-to-Video Diffusion Models with Temporal Consistency and Rigidity Constraintscited by 7
  10. 2026VAnim: Rendering-Aware Sparse State Modeling for Structure-Preserving Vector Animationcited by 6
  11. 2025Bezier Splatting for Fast and Differentiable Vector Graphics Renderingcited by 5
  12. 2025ViewCraft3D: High-Fidelity and View-Consistent 3D Vector Graphics Synthesiscited by 5
  13. 2025RewardSDS: Aligning Score Distillation via Reward-Weighted Samplingcited by 4
  14. 2026IntroSVG: Learning from Rendering Feedback for Text-to-SVG Generation via an Introspective Generator-Critic Frameworkcited by 4
  15. 2024CraftSVG: Multi-Object Text-to-SVG Synthesis via Layout Guided Diffusioncited by 3
  16. 2024SVGFusion: A VAE-Diffusion Transformer for Vector Graphic Generationcited by 3
  17. 2026Stroke of Surprise: Progressive Semantic Illusions in Vector Sketchingcited by 3
  18. 2026AmodalSVG: Amodal Image Vectorization via Semantic Layer Peelingcited by 3
  19. 2025AutoSketch: VLM-assisted Style-Aware Vector Sketch Completioncited by 2
  20. 2025StrokeFusion: Vector Sketch Generation via Joint Stroke-UDF Encoding and Latent Sequence Diffusioncited by 2
  21. 2025CoProSketch: Controllable and Progressive Sketch Generation with Diffusion Modelcited by 2
  22. 2024High-fidelity and Lip-synced Talking Face Synthesis via Landmark-based Diffusion Modelcited by 1
  23. 2024SketchRef: a Multi-Task Evaluation Benchmark for Sketch Synthesiscited by 1
  24. 2025Empowering Vector Graphics with Consistently Arbitrary Viewing and View-dependent Visibilitycited by 1
  25. 2026Teaching an Agent to Sketch One Part at a Timecited by 1
  26. 2026Hierarchical SVG Tokenization: Learning Compact Visual Programs for Scalable Vector Graphics Modelingcited by 1
  27. 2023CLIPDraw++: Text-to-Sketch Synthesis with Simple Primitivescited by 0
  28. 2025Recovering Partially Corrupted Objects via Sketch-Guided Bidirectional Feature Interactioncited by 0
  29. 2025SketchAnimator: Animate Sketch via Motion Customization of Text-to-Video Diffusion Modelscited by 0
  30. 2025Vector sketch animation generation with differentiable motion trajectoriescited by 0
  31. 20254-Doodle: Text to 3D Sketches that Move!cited by 0
  32. 2025Text to Sketch Generation with Multi-Stylescited by 0
  33. 2025DOGE: Differentiable Bezier Graph Optimization for Road Network Extractioncited by 0
  34. 2025Order Matters: 3D Shape Generation from Sequential VR Sketchescited by 0
  35. 2026Progressive Photorealistic Simplificationcited by 0
  36. 2026Vector Scaffolding: Inter-Scale Orchestration for Differentiable Image Vectorizationcited by 0
  37. 2026Optimizing 4D Wires for Sparse 3D Abstractioncited by 0
  38. 2026Single-Line Drawing Generation via Semantics-Driven Optimizationcited by 0

50 most cited of 54 All 54 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Training-Free Sketch-Guided Diffusion with Latent Optimization
  2. 2025SwiftSketch: A Diffusion Model for Image-to-Vector Sketch Generation
  3. 2025StableSketcher: Enhancing Diffusion Model for Pixel-based Sketch Generation via Visual Question Answering Feedback
  4. 2023SketchDreamer: Interactive Text-Augmented Creative Sketch Ideation
  5. 2023DiffSketching: Sketch Control Image Synthesis with Diffusion Models
  6. 2024Representative Feature Extraction During Diffusion Process for Sketch Extraction with One Example
  7. 2026SketchFlow: Zero-Shot Vector Sketch Generation via GMM Prior Flow in CLIP Latent Space
  8. 2022Sketch-Guided Text-to-Image Diffusion Models