# Text-to-Vector Generation with Neural Path Representation

arXiv: https://arxiv.org/abs/2405.10317 · HTML: https://mlsanity.com/paper/2405.10317/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 55 references that are in the map.
- First posted on arXiv 16 May 2024.

## Builds on (most cited first, 30 of 32)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [TokenFlow: Consistent Diffusion Features for Consistent Video Editing](https://mlsanity.com/paper/2307.10373/) (2023)
- [A Neural Representation of Sketch Drawings](https://mlsanity.com/paper/1704.03477/) (2017)
- [Inversion-Based Style Transfer with Diffusion Models](https://mlsanity.com/paper/2211.13203/) (2022)
- [CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders](https://mlsanity.com/paper/2106.14843/) (2021)
- [Sketch-Guided Text-to-Image Diffusion Models](https://mlsanity.com/paper/2211.13752/) (2022)
- [DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation](https://mlsanity.com/paper/2007.11301/) (2020)
- [Delta Denoising Score](https://mlsanity.com/paper/2304.07090/) (2023)
- [Im2Vec: Synthesizing Vector Graphics without Vector Supervision](https://mlsanity.com/paper/2102.02798/) (2021)
- [A Learned Representation for Scalable Vector Graphics](https://mlsanity.com/paper/1904.02632/) (2019)
- [Sketchformer: Transformer-based Representation for Sketched Structure](https://mlsanity.com/paper/2002.10381/) (2020)
- [SVGDreamer: Text Guided SVG Generation with Diffusion Model](https://mlsanity.com/paper/2312.16476/) (2023)
- [IconShop: Text-Guided Vector Icon Synthesis with Autoregressive Transformers](https://mlsanity.com/paper/2304.14400/) (2023)
- [Word-As-Image for Semantic Typography](https://mlsanity.com/paper/2303.01818/) (2023)
- [CLIPasso: Semantically-Aware Object Sketching](https://mlsanity.com/paper/2202.05822/) (2022)
- [FIGR: Few-shot Image Generation with Reptile](https://mlsanity.com/paper/1901.02199/) (2019)
- [Breathing Life Into Sketches Using Text-to-Video Priors](https://mlsanity.com/paper/2311.13608/) (2023)
- [CLIPVG: Text-Guided Image Manipulation Using Differentiable Vector Graphics](https://mlsanity.com/paper/2212.02122/) (2022)
- [StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Translation](https://mlsanity.com/paper/2202.12362/) (2022)
- [DeepVecFont-v2: Exploiting Transformers to Synthesize Vector Fonts with Higher Quality](https://mlsanity.com/paper/2303.14585/) (2023)
- [P-Cloth: Interactive Complex Cloth Simulation on Multi-GPU Systems using Dynamic Matrix Assembly and Pipelined Implicit Integrators](https://mlsanity.com/paper/2008.00409/) (2020)
- [DualVector: Unsupervised Vector Font Synthesis with Dual-Part Representation](https://mlsanity.com/paper/2305.10462/) (2023)

## Built on by (most cited first, 30 of 33)

- [Empowering LLMs to Understand and Generate Complex Vector Graphics](https://mlsanity.com/paper/2412.11102/) (2024)
- [LayerTracer: Cognitive-Aligned Layered SVG Synthesis via Diffusion Transformer](https://mlsanity.com/paper/2502.01105/) (2025)
- [Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Models](https://mlsanity.com/paper/2411.16602/) (2024)
- [SketchAgent: Language-Driven Sequential Sketch Generation](https://mlsanity.com/paper/2411.17673/) (2024)
- [Layered Image Vectorization via Semantic Simplification](https://mlsanity.com/paper/2406.05404/) (2024)
- [NeuralSVG: An Implicit Representation for Text-to-Vector Generation](https://mlsanity.com/paper/2501.03992/) (2025)
- [SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generation](https://mlsanity.com/paper/2506.03139/) (2025)
- [SVGen: Interpretable Vector Graphics Generation with Large Language Models](https://mlsanity.com/paper/2508.09168/) (2025)
- [Render-in-the-Loop: Vector Graphics Generation via Visual Self-Feedback](https://mlsanity.com/paper/2604.20730/) (2026)
- [Multi-Object Sketch Animation with Grouping and Motion Trajectory Priors](https://mlsanity.com/paper/2508.15535/) (2025)
- [LayerPeeler: Autoregressive Peeling for Layer-wise Image Vectorization](https://mlsanity.com/paper/2505.23740/) (2025)
- [DuetSVG: Unified Multimodal SVG Generation with Internal Visual Guidance](https://mlsanity.com/paper/2512.10894/) (2025)
- [VAnim: Rendering-Aware Sparse State Modeling for Structure-Preserving Vector Animation](https://mlsanity.com/paper/2605.01517/) (2026)
- [Bezier Splatting for Fast and Differentiable Vector Graphics Rendering](https://mlsanity.com/paper/2503.16424/) (2025)
- [Automatic Method Illustration Generation for AI Scientific Papers via Drawing Middleware Creation, Evolution, and Orchestration](https://mlsanity.com/paper/2603.29590/) (2026)
- [LottieGPT: Tokenizing Vector Animation for Autoregressive Generation](https://mlsanity.com/paper/2604.11792/) (2026)
- [Neural Image Abstraction Using Long Smoothing B-Splines](https://mlsanity.com/paper/2511.05360/) (2025)
- [IntroSVG: Learning from Rendering Feedback for Text-to-SVG Generation via an Introspective Generator-Critic Framework](https://mlsanity.com/paper/2603.09312/) (2026)
- [SVGFusion: A VAE-Diffusion Transformer for Vector Graphic Generation](https://mlsanity.com/paper/2412.10437/) (2024)
- [Stroke of Surprise: Progressive Semantic Illusions in Vector Sketching](https://mlsanity.com/paper/2602.12280/) (2026)
- [From Fragment to One Piece: A Survey on AI-Driven Graphic Design](https://mlsanity.com/paper/2503.18641/) (2025)
- [Style Customization of Text-to-Vector Generation with Image Diffusion Priors](https://mlsanity.com/paper/2505.10558/) (2025)
- [Vector Prism: Animating Vector Graphics by Stratifying Semantic Structure](https://mlsanity.com/paper/2512.14336/) (2025)
- [Empowering Vector Graphics with Consistently Arbitrary Viewing and View-dependent Visibility](https://mlsanity.com/paper/2505.21377/) (2025)
- [Illustrator's Depth: Monocular Layer Index Prediction for Image Decomposition](https://mlsanity.com/paper/2511.17454/) (2025)
- [Structural Evaluation Metrics for SVG Generation via Leave-One-Out Analysis](https://mlsanity.com/paper/2604.08809/) (2026)
- [GVR-Coder: A Visual-Feedback Framework for Structured SVG Generation in Complex Document and Meeting Scenarios](https://mlsanity.com/paper/2607.28073/) (2026)
- [A Scalable Vector Graphics Latent Space](https://mlsanity.com/paper/2608.21893/) (2026)
- [DesigNet: Learning to Draw Vector Graphics as Designers Do](https://mlsanity.com/paper/2604.06494/) (2026)
- [Vector Scaffolding: Inter-Scale Orchestration for Differentiable Image Vectorization](https://mlsanity.com/paper/2605.11913/) (2026)

## Abstract (arXiv)

Vector graphics are widely used in digital art and highly favored by designers due to their scalability and layer-wise properties. However, the process of creating and editing vector graphics requires creativity and design expertise, making it a time-consuming task. Recent advancements in text-to-vector (T2V) generation have aimed to make this process more accessible. However, existing T2V methods directly optimize control points of vector graphics paths, often resulting in intersecting or jagged paths due to the lack of geometry constraints. To overcome these limitations, we propose a novel neural path representation by designing a dual-branch Variational Autoencoder (VAE) that learns the path latent space from both sequence and image modalities. By optimizing the combination of neural paths, we can incorporate geometric constraints while preserving expressivity in generated SVGs. Furthermore, we introduce a two-stage path optimization method to improve the visual and topological quality of generated SVGs. In the first stage, a pre-trained text-to-image diffusion model guides the initial generation of complex vector graphics through the Variational Score Distillation (VSD) process. In the second stage, we refine the graphics using a layer-wise image vectorization strategy to achieve clearer elements and structure. We demonstrate the effectiveness of our method through extensive experiments and showcase various applications. The project page is https://intchous.github.io/T2V-NPR.