Render-in-the-Loop: Vector Graphics Generation via Visual Self-Feedback
On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection
Abstract (arXiv)
Multimodal Large Language Models (MLLMs) have shown promising capabilities in generating Scalable Vector Graphics (SVG) via direct code synthesis. However, existing paradigms typically adopt an open-loop "blind drawing" approach, where models generate symbolic code sequences without perceiving intermediate visual outcomes. This methodology severely underutilizes the powerful visual priors embedded in MLLMs vision encoders, treating SVG generation as a disjointed textual sequence modeling task rather than an integrated visuo-spatial one. Consequently, models struggle to reason about partial canvas states and implicit occlusion relationships, which are visually explicit but textually ambiguous. To bridge this gap, we propose Render-in-the-Loop, a novel generation paradigm that reformulates SVG synthesis as a step-wise, visual-context-aware process. By rendering intermediate code states into a cumulative canvas, the model explicitly observes the evolving visual context at each step, leveraging on-the-fly feedback to guide subsequent generation. However, we demonstrate that applying this visual loop naively to off-the-shelf models is suboptimal due to their inability to leverage incremental visual-code mappings. To address this, we first utilize fine-grained path decomposition to construct dense multi-step visual trajectories, and then introduce a Visual Self-Feedback (VSF) training strategy to condition the next primitive generation on intermediate visual states. Furthermore, a Render-and-Verify (RaV) inference mechanism is proposed to effectively filter degenerate and redundant primitives. Our framework, instantiated on a multimodal foundation model, outperforms strong open-weight baselines on the standard MMSVGBench. This result highlights the remarkable data efficiency and generalization capability of our Render-in-the-Loop paradigm for both Text-to-SVG and Image-to-SVG tasks.
Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 58 of its 84 references that are in the map. First posted on arXiv 22 Apr 2026.
Builds on
58 of 84 references.
Closest ancestors · 3
Same direction, newest first
- 2026Reliable Reasoning in SVG-LLMs via Multi-Task Multi-Reward Reinforcement Learning17 Mar
- 2025DuetSVG: Unified Multimodal SVG Generation with Internal Visual Guidance11 Dec
- 2025InternSVG: Towards Unified SVG Tasks with Multimodal Large Language Models13 Oct
Other references · 55
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2023GPT-4 Technical Reportcited by 18,132
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
- 2017A Neural Representation of Sketch Drawingscited by 311
38 more
- 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preferencecited by 284
- 2021CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoderscited by 186
- 2020DeepSVG: A Hierarchical Generative Network for Vector Graphics Animationcited by 158
- 2021Im2Vec: Synthesizing Vector Graphics without Vector Supervisioncited by 114
- 2022VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Modelscited by 112
- 2023StarVector: Generating Scalable Vector Graphics Code from Images and Textcited by 96
- 2019A Learned Representation for Scalable Vector Graphicscited by 91
- 2022Towards Layer-wise Image Vectorizationcited by 77
- 2023SVGDreamer: Text Guided SVG Generation with Diffusion Modelcited by 75
- 2023IconShop: Text-Guided Vector Icon Synthesis with Autoregressive Transformerscited by 73
- 2025OmniSVG: A Unified Scalable Vector Graphics Generation Modelcited by 71
- 2022CLIPascene: Scene Sketching with Different Types and Levels of Abstractioncited by 63
- 2024Empowering LLMs to Understand and Generate Complex Vector Graphicscited by 63
- 2023Word-As-Image for Semantic Typographycited by 62
- 2023DiffSketcher: Text Guided Vector Sketch Synthesis through Latent Diffusion Modelscited by 54
- 2025Frequency-Aligned Knowledge Distillation for Lightweight Spatiotemporal Forecastingcited by 53
- 2023Breathing Life Into Sketches Using Text-to-Video Priorscited by 47
- 2023RegFormer: An Efficient Projection-Aware Transformer Network for Large-Scale Point Cloud Registrationcited by 37
- 2024Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Modelscited by 36
- 2025Rendering-Aware Reinforcement Learning for Vector Graphics Generationcited by 36
- 2024Text-to-Vector Generation with Neural Path Representationcited by 32
- 2024SketchAgent: Language-Driven Sequential Sketch Generationcited by 30
- 2024StrokeNUWA: Tokenizing Strokes for Vector Graphic Synthesiscited by 26
- 2024Swift Sampler: Efficient Learning of Sampler by 10 Parameterscited by 25
- 2024SuperSVG: Superpixel-based Scalable Vector Graphics Synthesiscited by 19
- 2024Layered Image Vectorization via Semantic Simplificationcited by 18
- 2024NIVeL: Neural Implicit Vector Layers for Text-to-Vector Generationcited by 17
- 2025Uni-Layout: Integrating Human Feedback in Unified Layout Generation and Evaluationcited by 13
- 2025SVGen: Interpretable Vector Graphics Generation with Large Language Modelscited by 13
- 2024Segmentation-guided Layer-wise Image Vectorization with Gradient Fillscited by 11
- 2025AutoPCR: Automated Phenotype Concept Recognition by Promptingcited by 10
- 2025Multi-Object Sketch Animation with Grouping and Motion Trajectory Priorscited by 9
- 2025LayerPeeler: Autoregressive Peeling for Layer-wise Image Vectorizationcited by 8
- 2025SVGThinker: Instruction-Aligned and Reasoning-Driven Text-to-SVG Generationcited by 8
- 2025Quantized-Tinyllava: a new multimodal foundation model enables efficient split learningcited by 8
- 2023HarmonicNeRF: Geometry-Informed Synthetic View Augmentation for 3D Scene Reconstruction in Driving Scenarioscited by 7
- 2024VectorPainter: Advanced Stylized Vector Graphics Synthesis Using Stroke-Style Priorscited by 7
- 2023SAMVG: A Multi-stage Image Vectorization Model with the Segment-Anything Modelcited by 6
50 most cited of 55 All 55 (JSON)
Built on by
Citing papers per year: 2026: 12.
12 papers.
- 2026MedSynapse-V: Bridging Visual Perception and Clinical Intuition via Latent Memory Evolutioncited by 12
- 2026VAnim: Rendering-Aware Sparse State Modeling for Structure-Preserving Vector Animationcited by 6
- 2026MIS-HCC: Hierarchical Channel Clustering for Efficient Medical Image Segmentationcited by 4
- 2026Curvature-Adaptive Consistency Flow Matching: Autonomous Trajectory Optimization via Reinforcement Learningcited by 3
- 2026Deformba: Vision State Space Model with Adaptive State Fusioncited by 2
- 2026BoostAPR: Boosting Automated Program Repair via Execution-Grounded Reinforcement Learning with Dual Reward Modelscited by 1
- 2026SynGR: Unleashing the Potential of Cross-Modal Synergy for Generative Recommendationcited by 1
- 2026Modality-Decoupled Online Recursive Editingcited by 1
- 2026Semantic Granularity Navigation in Image Editingcited by 1
- 2026CANVAS: Consistency-Aware Navigation via Visual Adaptive Sampling for Long-Context Text-to-SVG Generationcited by 0
- 2026Agentic Visual Generation: From Generative Models to Agentic Controlcited by 0
- 2026Reinforcement Learning from Intermediate Renders for Image-to-Code Generationcited by 0
- 2026Reinforcement Learning from Intermediate Renders for Image-to-Code Generation28 Sep 2026
- 2026Agentic Visual Generation: From Generative Models to Agentic Control6 Sep 2026
- 2026CANVAS: Consistency-Aware Navigation via Visual Adaptive Sampling for Long-Context Text-to-SVG Generation31 Aug 2026
- 2026MIS-HCC: Hierarchical Channel Clustering for Efficient Medical Image Segmentation19 Jul 2026
- 2026Curvature-Adaptive Consistency Flow Matching: Autonomous Trajectory Optimization via Reinforcement Learning21 Jun 2026
- 2026Deformba: Vision State Space Model with Adaptive State Fusion20 May 2026
- 2026Semantic Granularity Navigation in Image Editing20 May 2026
- 2026Modality-Decoupled Online Recursive Editing19 May 2026
- 2026SynGR: Unleashing the Potential of Cross-Modal Synergy for Generative Recommendation18 May 2026
- 2026BoostAPR: Boosting Automated Program Repair via Execution-Grounded Reinforcement Learning with Dual Reward Models9 May 2026
- 2026VAnim: Rendering-Aware Sparse State Modeling for Structure-Preserving Vector Animation2 May 2026
- 2026MedSynapse-V: Bridging Visual Perception and Clinical Intuition via Latent Memory Evolution29 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2026Evaluating Constrained Iterative Refinement for Scalable Vector Graphics Generation with Off-the-Shelf VLMs
- 2026RefineSVG: Visual Feedback-Driven Reinforcement Learning for Image-to-SVG Generation
- 2026IntroSVG: Learning from Rendering Feedback for Text-to-SVG Generation via an Introspective Generator-Critic Framework
- 2025Vector Prism: Animating Vector Graphics by Stratifying Semantic Structure
- 2024SVGFusion: A VAE-Diffusion Transformer for Vector Graphic Generation
- 2026VectorGym: A Multi-Task Benchmark for SVG Code Generation, Sketching and Editing
- 2024VGBench: Evaluating Large Language Models on Vector Graphics Understanding and Generation
- 2024CraftSVG: Multi-Object Text-to-SVG Synthesis via Layout Guided Diffusion