mlsanityData to 7 Oct 2026

VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models

arXiv 2211.11319 ↗First posted 21 Nov 2022cs.CV cs.AI cs.GRcited by 112

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Diffusion models have shown impressive results in text-to-image synthesis. Using massive datasets of captioned images, diffusion models learn to generate raster images of highly diverse objects and scenes. However, designers frequently use vector representations of images like Scalable Vector Graphics (SVGs) for digital icons or art. Vector graphics can be scaled to any size, and are compact. We show that a text-conditioned diffusion model trained on pixel representations of images can be used to generate SVG-exportable vector graphics. We do so without access to large datasets of captioned SVGs. By optimizing a differentiable vector graphics rasterizer, our method, VectorFusion, distills abstract semantic knowledge out of a pretrained diffusion model. Inspired by recent text-to-3D work, we learn an SVG consistent with a caption using Score Distillation Sampling. To accelerate generation and improve fidelity, VectorFusion also initializes from an image sample. Experiments show greater quality than prior work, and demonstrate a range of styles including pixel art and sketches. See our project webpage at https://ajayj.com/vectorfusion .

Key facts. Cited by 112 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 36 of its 45 references that are in the map. First posted on arXiv 21 Nov 2022.

Builds on

36 of 45 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  3. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2020Denoising Diffusion Implicit Modelscited by 9,655
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  10. 2022Classifier-Free Diffusion Guidancecited by 5,717
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  12. 2013Auto-Encoding Variational Bayescited by 3,980
21 more
  1. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  2. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  3. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  4. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  5. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  6. 2021Variational Diffusion Modelscited by 1,145
  7. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 695
  8. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  9. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  10. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  11. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 326
  12. 2017A Neural Representation of Sketch Drawingscited by 311
  13. 2019Kornia: an Open Source Differentiable Computer Vision Library for PyTorchcited by 231
  14. 2022Diffusion models as plug-and-play priorscited by 230
  15. 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214
  16. 2021CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoderscited by 186
  17. 2022Towards Layer-wise Image Vectorizationcited by 77
  18. 2021Modern Evolution Strategies for Creativity: Fitting Concrete Images and Abstract Conceptscited by 40
  19. 2022StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Translationcited by 28
  20. 2022CLIP-CLOP: CLIP-Guided Collage and Photomontagecited by 11
  21. 2021Generative Art Using Neural Visual Grammars and Dual Encoderscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 29; 2024: 27; 2025: 28; 2026: 28.

112 papers.

Sort:

By area:

  1. 2023Text-to-image Diffusion Models in Generative AI: A Surveycited by 264
  2. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  3. 2023DemoFusion: Democratising High-Resolution Image Generation With No cited by 105
  4. 2023Noise-Free Score Distillationcited by 99
  5. 2023StarVector: Generating Scalable Vector Graphics Code from Images and Textcited by 96
  6. 2023SVGDreamer: Text Guided SVG Generation with Diffusion Modelcited by 75
  7. 2023AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZcited by 74
  8. 2023IconShop: Text-Guided Vector Icon Synthesis with Autoregressive Transformerscited by 73
  9. 2023Image Denoising: The Deep Learning Revolution and Beyond -- A Survey Paper --cited by 71
  10. 2025OmniSVG: A Unified Scalable Vector Graphics Generation Modelcited by 71
  11. 2024Empowering LLMs to Understand and Generate Complex Vector Graphicscited by 63
  12. 2023Word-As-Image for Semantic Typographycited by 62
38 more
  1. 2023DiffSketcher: Text Guided Vector Sketch Synthesis through Latent Diffusion Modelscited by 54
  2. 2023Breathing Life Into Sketches Using Text-to-Video Priorscited by 47
  3. 2023Posterior Distillation Samplingcited by 45
  4. 2023Prompting Hard or Hardly Prompting: Prompt Inversion for Text-to-Image Diffusion Modelscited by 40
  5. 2023COLE: A Hierarchical Generation Framework for Multi-Layered and Editable Graphic Designcited by 39
  6. 2023NOPE: Novel Object Pose Estimation from a Single Imagecited by 38
  7. 2023Farm3D: Learning Articulated 3D Animals by Distilling 2D Diffusioncited by 37
  8. 2024Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Modelscited by 36
  9. 2023Taming Mode Collapse in Score Distillation for Text-to-3D Generationcited by 35
  10. 2024PALP: Prompt Aligned Personalization of Text-to-Image Modelscited by 35
  11. 2024SketchAgent: Language-Driven Sequential Sketch Generationcited by 30
  12. 2024Score Distillation Sampling with Learned Manifold Correctivecited by 27
  13. 2024StrokeNUWA: Tokenizing Strokes for Vector Graphic Synthesiscited by 26
  14. 2024SyncTweedies: A General Generative Framework Based on Synchronized Diffusionscited by 26
  15. 2024SVGEditBench: A Benchmark Dataset for Quantitative Assessment of LLM's SVG Editing Capabilitiescited by 25
  16. 2023As-Plausible-As-Possible: Plausibility-Aware Mesh Deformation Using 2D Diffusion Priorscited by 24
  17. 2024DreamSampler: Unifying Diffusion Sampling and Score Distillation for Image Manipulationcited by 22
  18. 2025TikZero: Zero-Shot Text-Guided Graphics Program Synthesiscited by 21
  19. 2024Alfie: Democratising RGBA Image Generation With No cited by 20
  20. 2024Layered Image Vectorization via Semantic Simplificationcited by 18
  21. 2024AniClipart: Clipart Animation with Text-to-Video Priorscited by 17
  22. 2024NIVeL: Neural Implicit Vector Layers for Text-to-Vector Generationcited by 17
  23. 2024Diffusion-Based Visual Art Creation: A Survey and New Perspectivescited by 17
  24. 2023Wired Perspectives: Multi-View Wire Art Embraces Generative AIcited by 16
  25. 2025NeuralSVG: An Implicit Representation for Text-to-Vector Generationcited by 16
  26. 2024SVGBuilder: Component-Based Colored SVG Generation with Text-Guided Autoregressive Transformerscited by 14
  27. 2023CLIPAG: Towards Generator-Free Text-to-Image Generationcited by 12
  28. 2025InternSVG: Towards Unified SVG Tasks with Multimodal Large Language Modelscited by 12
  29. 2026Render-in-the-Loop: Vector Graphics Generation via Visual Self-Feedbackcited by 12
  30. 2023SketchDreamer: Interactive Text-Augmented Creative Sketch Ideationcited by 10
  31. 2024Diff3DS: Generating View-Consistent 3D Sketch via Differentiable Curve Renderingcited by 9
  32. 2025OracleFusion: Assisting the Decipherment of Oracle Bone Script with Structurally Constrained Semantic Typographycited by 8
  33. 2023Image Vectorization: a Reviewcited by 7
  34. 2024VectorPainter: Advanced Stylized Vector Graphics Synthesis Using Stroke-Style Priorscited by 7
  35. 2024Enhancing Sketch Animation: Text-to-Video Diffusion Models with Temporal Consistency and Rigidity Constraintscited by 7
  36. 2025Rethinking Score Distilling Sampling for 3D Editing and Generationcited by 7
  37. 2023Controlled and Conditional Text to Image Generation with Diffusion Priorcited by 6
  38. 2025Symbolic Graphics Programming with Large Language Modelscited by 6

50 most cited of 112 All 112 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025SVG-T2I: Scaling Up Text-to-Image Latent Diffusion Model Without Variational Autoencoder
  2. 2025Style Customization of Text-to-Vector Generation with Image Diffusion Priors
  3. 2026VFIG: Vectorizing Complex Figures in SVG with Vision-Language Models
  4. 2023Beyond Pixels: Exploring Human-Readable SVG Generation for Simple Images with Vision Language Models
  5. 2025SVGThinker: Instruction-Aligned and Reasoning-Driven Text-to-SVG Generation
  6. 2025Latent Diffusion Model without Variational Autoencoder
  7. 2026VectorGym: A Multi-Task Benchmark for SVG Code Generation, Sketching and Editing
  8. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesis