mlsanityData to 7 Oct 2026

Render-in-the-Loop: Vector Graphics Generation via Visual Self-Feedback

arXiv 2604.20730 ↗First posted 22 Apr 2026cs.CVcited by 12

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have shown promising capabilities in generating Scalable Vector Graphics (SVG) via direct code synthesis. However, existing paradigms typically adopt an open-loop "blind drawing" approach, where models generate symbolic code sequences without perceiving intermediate visual outcomes. This methodology severely underutilizes the powerful visual priors embedded in MLLMs vision encoders, treating SVG generation as a disjointed textual sequence modeling task rather than an integrated visuo-spatial one. Consequently, models struggle to reason about partial canvas states and implicit occlusion relationships, which are visually explicit but textually ambiguous. To bridge this gap, we propose Render-in-the-Loop, a novel generation paradigm that reformulates SVG synthesis as a step-wise, visual-context-aware process. By rendering intermediate code states into a cumulative canvas, the model explicitly observes the evolving visual context at each step, leveraging on-the-fly feedback to guide subsequent generation. However, we demonstrate that applying this visual loop naively to off-the-shelf models is suboptimal due to their inability to leverage incremental visual-code mappings. To address this, we first utilize fine-grained path decomposition to construct dense multi-step visual trajectories, and then introduce a Visual Self-Feedback (VSF) training strategy to condition the next primitive generation on intermediate visual states. Furthermore, a Render-and-Verify (RaV) inference mechanism is proposed to effectively filter degenerate and redundant primitives. Our framework, instantiated on a multimodal foundation model, outperforms strong open-weight baselines on the standard MMSVGBench. This result highlights the remarkable data efficiency and generalization capability of our Render-in-the-Loop paradigm for both Text-to-SVG and Image-to-SVG tasks.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 58 of its 84 references that are in the map. First posted on arXiv 22 Apr 2026.

Builds on

58 of 84 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Reliable Reasoning in SVG-LLMs via Multi-Task Multi-Reward Reinforcement Learning17 Mar
  2. 2025DuetSVG: Unified Multimodal SVG Generation with Internal Visual Guidance11 Dec
  3. 2025InternSVG: Towards Unified SVG Tasks with Multimodal Large Language Models13 Oct

Other references · 55

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2017Proximal Policy Optimization Algorithmscited by 14,441
  7. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  8. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  9. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  10. 2025Qwen3-VL Technical Reportcited by 2,534
  11. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
  12. 2017A Neural Representation of Sketch Drawingscited by 311
38 more
  1. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preferencecited by 284
  2. 2021CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoderscited by 186
  3. 2020DeepSVG: A Hierarchical Generative Network for Vector Graphics Animationcited by 158
  4. 2021Im2Vec: Synthesizing Vector Graphics without Vector Supervisioncited by 114
  5. 2022VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Modelscited by 112
  6. 2023StarVector: Generating Scalable Vector Graphics Code from Images and Textcited by 96
  7. 2019A Learned Representation for Scalable Vector Graphicscited by 91
  8. 2022Towards Layer-wise Image Vectorizationcited by 77
  9. 2023SVGDreamer: Text Guided SVG Generation with Diffusion Modelcited by 75
  10. 2023IconShop: Text-Guided Vector Icon Synthesis with Autoregressive Transformerscited by 73
  11. 2025OmniSVG: A Unified Scalable Vector Graphics Generation Modelcited by 71
  12. 2022CLIPascene: Scene Sketching with Different Types and Levels of Abstractioncited by 63
  13. 2024Empowering LLMs to Understand and Generate Complex Vector Graphicscited by 63
  14. 2023Word-As-Image for Semantic Typographycited by 62
  15. 2023DiffSketcher: Text Guided Vector Sketch Synthesis through Latent Diffusion Modelscited by 54
  16. 2025Frequency-Aligned Knowledge Distillation for Lightweight Spatiotemporal Forecastingcited by 53
  17. 2023Breathing Life Into Sketches Using Text-to-Video Priorscited by 47
  18. 2023RegFormer: An Efficient Projection-Aware Transformer Network for Large-Scale Point Cloud Registrationcited by 37
  19. 2024Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Modelscited by 36
  20. 2025Rendering-Aware Reinforcement Learning for Vector Graphics Generationcited by 36
  21. 2024Text-to-Vector Generation with Neural Path Representationcited by 32
  22. 2024SketchAgent: Language-Driven Sequential Sketch Generationcited by 30
  23. 2024StrokeNUWA: Tokenizing Strokes for Vector Graphic Synthesiscited by 26
  24. 2024Swift Sampler: Efficient Learning of Sampler by 10 Parameterscited by 25
  25. 2024SuperSVG: Superpixel-based Scalable Vector Graphics Synthesiscited by 19
  26. 2024Layered Image Vectorization via Semantic Simplificationcited by 18
  27. 2024NIVeL: Neural Implicit Vector Layers for Text-to-Vector Generationcited by 17
  28. 2025Uni-Layout: Integrating Human Feedback in Unified Layout Generation and Evaluationcited by 13
  29. 2025SVGen: Interpretable Vector Graphics Generation with Large Language Modelscited by 13
  30. 2024Segmentation-guided Layer-wise Image Vectorization with Gradient Fillscited by 11
  31. 2025AutoPCR: Automated Phenotype Concept Recognition by Promptingcited by 10
  32. 2025Multi-Object Sketch Animation with Grouping and Motion Trajectory Priorscited by 9
  33. 2025LayerPeeler: Autoregressive Peeling for Layer-wise Image Vectorizationcited by 8
  34. 2025SVGThinker: Instruction-Aligned and Reasoning-Driven Text-to-SVG Generationcited by 8
  35. 2025Quantized-Tinyllava: a new multimodal foundation model enables efficient split learningcited by 8
  36. 2023HarmonicNeRF: Geometry-Informed Synthetic View Augmentation for 3D Scene Reconstruction in Driving Scenarioscited by 7
  37. 2024VectorPainter: Advanced Stylized Vector Graphics Synthesis Using Stroke-Style Priorscited by 7
  38. 2023SAMVG: A Multi-stage Image Vectorization Model with the Segment-Anything Modelcited by 6

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 12.

12 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Evaluating Constrained Iterative Refinement for Scalable Vector Graphics Generation with Off-the-Shelf VLMs
  2. 2026RefineSVG: Visual Feedback-Driven Reinforcement Learning for Image-to-SVG Generation
  3. 2026IntroSVG: Learning from Rendering Feedback for Text-to-SVG Generation via an Introspective Generator-Critic Framework
  4. 2025Vector Prism: Animating Vector Graphics by Stratifying Semantic Structure
  5. 2024SVGFusion: A VAE-Diffusion Transformer for Vector Graphic Generation
  6. 2026VectorGym: A Multi-Task Benchmark for SVG Code Generation, Sketching and Editing
  7. 2024VGBench: Evaluating Large Language Models on Vector Graphics Understanding and Generation
  8. 2024CraftSVG: Multi-Object Text-to-SVG Synthesis via Layout Guided Diffusion