# VectorHarness: Recovering Editable, Relation-Preserving Structure from Scientific Graphics

arXiv: https://arxiv.org/abs/2609.13294 · HTML: https://mlsanity.com/paper/2609.13294/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 45 references that are in the map.
- First posted on arXiv 9 Sep 2026.

## Builds on (most cited first, 30 of 34)

- [DePlot: One-shot visual language reasoning by plot-to-table translation](https://mlsanity.com/paper/2212.10505/) (2022)
- [DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation](https://mlsanity.com/paper/2007.11301/) (2020)
- [Im2Vec: Synthesizing Vector Graphics without Vector Supervision](https://mlsanity.com/paper/2102.02798/) (2021)
- [VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models](https://mlsanity.com/paper/2211.11319/) (2022)
- [StarVector: Generating Scalable Vector Graphics Code from Images and Text](https://mlsanity.com/paper/2312.11556/) (2023)
- [ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation](https://mlsanity.com/paper/2501.06598/) (2025)
- [Image-to-Markup Generation with Coarse-to-Fine Attention](https://mlsanity.com/paper/1609.04938/) (2016)
- [Towards Layer-wise Image Vectorization](https://mlsanity.com/paper/2206.04655/) (2022)
- [SVGDreamer: Text Guided SVG Generation with Diffusion Model](https://mlsanity.com/paper/2312.16476/) (2023)
- [MakeAnything: Harnessing Diffusion Transformers for Multi-Domain Procedural Sequence Generation](https://mlsanity.com/paper/2502.01572/) (2025)
- [LayerTracer: Cognitive-Aligned Layered SVG Synthesis via Diffusion Transformer](https://mlsanity.com/paper/2502.01105/) (2025)
- [FonTS: Text Rendering with Typography and Style Controls](https://mlsanity.com/paper/2412.00136/) (2024)
- [Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Models](https://mlsanity.com/paper/2411.16602/) (2024)
- [CLIPVG: Text-Guided Image Manipulation Using Differentiable Vector Graphics](https://mlsanity.com/paper/2212.02122/) (2022)
- [OmniConsistency: Learning Style-Agnostic Consistency from Paired Stylization Data](https://mlsanity.com/paper/2505.18445/) (2025)
- [EasyText: Controllable Diffusion Transformer for Multilingual Text Rendering](https://mlsanity.com/paper/2505.24417/) (2025)
- [RelationAdapter: Learning and Transferring Visual Relation with Diffusion Transformers](https://mlsanity.com/paper/2506.02528/) (2025)
- [LayerD: Decomposing Raster Graphic Designs into Layers](https://mlsanity.com/paper/2509.25134/) (2025)
- [DiffSim: Taming Diffusion Models for Evaluating Visual Similarity](https://mlsanity.com/paper/2412.14580/) (2024)
- [DiffDecompose: Layer-Wise Decomposition of Alpha-Composited Images via Diffusion Transformers](https://mlsanity.com/paper/2505.21541/) (2025)
- [SuperSVG: Superpixel-based Scalable Vector Graphics Synthesis](https://mlsanity.com/paper/2406.09794/) (2024)
- [WordCon: Word-level Typography Control in Scene Text Rendering](https://mlsanity.com/paper/2506.21276/) (2025)
- [Draw with Thought: Unleashing Multimodal Reasoning for Scientific Diagram Generation](https://mlsanity.com/paper/2504.09479/) (2025)
- [AutoFigure-Edit: Generating Editable Scientific Illustration](https://mlsanity.com/paper/2603.06674/) (2026)
- [From Charts to Code: A Hierarchical Benchmark for Multimodal Models](https://mlsanity.com/paper/2510.17932/) (2025)
- [MCA-Bench: A Multimodal Benchmark for Evaluating CAPTCHA Robustness Against VLM-based Attacks](https://mlsanity.com/paper/2506.05982/) (2025)
- [CreatiParser: Generative Image Parsing of Raster Graphic Designs into Editable Layers](https://mlsanity.com/paper/2604.19632/) (2026)
- [Crafter: A Multi-Agent Harness for Editable Scientific Figure Generation from Diverse Inputs](https://mlsanity.com/paper/2605.30611/) (2026)
- [Semantic Document Derendering: SVG Reconstruction via Vision-Language Modeling](https://mlsanity.com/paper/2511.13478/) (2025)
- [VFIG: Vectorizing Complex Figures in SVG with Vision-Language Models](https://mlsanity.com/paper/2603.24575/) (2026)

## Abstract (arXiv)

Converting scientific graphics into editable representations remains a challenging problem for image-to-code generation because of their heterogeneous elements and complex layouts. Recent multi-agent reconstruction systems have advanced this line of work, but often follow a copy-paste paradigm: the reconstructed image closely resembles the original, while complex regions remain effectively uneditable. We instead formulate a different objective, raster-to-authoring reconstruction, which aims to recover an authoring representation that supports native, customized editing rather than mere visual replication. To this end, we present VectorHarness, a multi-agent framework for raster-to-authoring reconstruction that recovers heterogeneous components using type-appropriate native representations. Text, formulas, shapes, connectors, icons, charts, and tables are reconstructed as natively editable objects, while intrinsically image-based regions remain raster content. To systematically evaluate reconstruction quality, we introduce VectorHarness-Bench, which jointly assesses rendering fidelity, raster fallback coverage, executable object edits, and relation-preserving edits. Experiments show that VectorHarness improves executable edit success and relation preservation, reduces avoidable raster fallback, and maintains high visual fidelity across heterogeneous graphics.