# VectorGym: A Multi-Task Benchmark for SVG Code Generation, Sketching and Editing

arXiv: https://arxiv.org/abs/2603.29852 · HTML: https://mlsanity.com/paper/2603.29852/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 42 references that are in the map.
- First posted on arXiv 22 Feb 2026.

## Builds on (most cited first, 14 of 14)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation](https://mlsanity.com/paper/2007.11301/) (2020)
- [StarVector: Generating Scalable Vector Graphics Code from Images and Text](https://mlsanity.com/paper/2312.11556/) (2023)
- [A Learned Representation for Scalable Vector Graphics](https://mlsanity.com/paper/1904.02632/) (2019)
- [AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZ](https://mlsanity.com/paper/2310.00367/) (2023)
- [OmniSVG: A Unified Scalable Vector Graphics Generation Model](https://mlsanity.com/paper/2504.06263/) (2025)
- [Empowering LLMs to Understand and Generate Complex Vector Graphics](https://mlsanity.com/paper/2412.11102/) (2024)
- [FIGR: Few-shot Image Generation with Reptile](https://mlsanity.com/paper/1901.02199/) (2019)
- [Rendering-Aware Reinforcement Learning for Vector Graphics Generation](https://mlsanity.com/paper/2505.20793/) (2025)
- [OCR-VQGAN: Taming Text-within-Image Generation](https://mlsanity.com/paper/2210.11248/) (2022)
- [SVGEditBench: A Benchmark Dataset for Quantitative Assessment of LLM's SVG Editing Capabilities](https://mlsanity.com/paper/2404.13710/) (2024)
- [FigGen: Text to Scientific Figure Generation](https://mlsanity.com/paper/2306.00800/) (2023)
- [SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generation](https://mlsanity.com/paper/2506.03139/) (2025)
- [BigDocs: An Open Dataset for Training Multimodal Models on Document and Code Tasks](https://mlsanity.com/paper/2412.04626/) (2024)

## Built on by (most cited first, 5 of 5)

- [Vision2Code: A Multi-Domain Benchmark for Evaluating Image-to-Code Generation](https://mlsanity.com/paper/2605.11307/) (2026)
- [GeoSVG-RL: Geometry-Aware Reinforcement Learning for Layout-Constrained Text-to-SVG Diagram Generation](https://mlsanity.com/paper/2605.25447/) (2026)
- [RULER: Instance-aware Rubric Rewards for SVG Generation](https://mlsanity.com/paper/2609.25270/) (2026)
- [SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation](https://mlsanity.com/paper/2609.03806/) (2026)
- [Compositional SVG Generation via VLM-Driven Hierarchical Semantic Parsing](https://mlsanity.com/paper/2609.14657/) (2026)

## Abstract (arXiv)

We introduce VectorGym, a comprehensive benchmark suite for Scalable Vector Graphics (SVG) that spans generation from text and sketches, complex editing, and visual understanding. VectorGym addresses the lack of realistic, challenging benchmarks aligned with professional design workflows. Our benchmark comprises four tasks with expert human-authored annotations: the novel Sketch2SVG task (VG-Sketch); a new SVG editing dataset (VG-Edit) featuring complex, multi-step edits with higher-order primitives; Text2SVG generation (VG-Text); and SVG captioning (VG-Cap). Unlike prior benchmarks that rely on synthetic edits, VectorGym provides gold-standard human annotations that require semantic understanding and design intent. We also provide a multi-task reinforcement learning baseline that jointly optimizes across all four tasks using rendering-based rewards. This baseline, built on GRPO with curriculum learning, trains a Qwen3-VL 8B model that achieves state-of-the-art performance among open-source models, surpassing much larger models including Qwen3-VL 235B and matching GPT-4o. We also introduce a VLM-as-a-Judge metric for SVG generation, validated through human correlation studies. Our evaluation of frontier VLMs reveals significant performance gaps, positioning VectorGym as a rigorous framework for advancing visual code generation.