# Layered Image Vectorization via Semantic Simplification

arXiv: https://arxiv.org/abs/2406.05404 · HTML: https://mlsanity.com/paper/2406.05404/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 69 references that are in the map.
- First posted on arXiv 8 Jun 2024.

## Builds on (most cited first, 22 of 22)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching](https://mlsanity.com/paper/2311.11284/) (2023)
- [Im2Vec: Synthesizing Vector Graphics without Vector Supervision](https://mlsanity.com/paper/2102.02798/) (2021)
- [Transparent Image Layer Diffusion using Latent Transparency](https://mlsanity.com/paper/2402.17113/) (2024)
- [VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models](https://mlsanity.com/paper/2211.11319/) (2022)
- [Single Image Reflection Separation with Perceptual Losses](https://mlsanity.com/paper/1806.05376/) (2018)
- [Towards Layer-wise Image Vectorization](https://mlsanity.com/paper/2206.04655/) (2022)
- [IconShop: Text-Guided Vector Icon Synthesis with Autoregressive Transformers](https://mlsanity.com/paper/2304.14400/) (2023)
- [Stylized Neural Painting](https://mlsanity.com/paper/2011.08114/) (2020)
- [Single Image Reflection Separation via Component Synergy](https://mlsanity.com/paper/2308.10027/) (2023)
- [Text-to-Vector Generation with Neural Path Representation](https://mlsanity.com/paper/2405.10317/) (2024)
- [Deep Vectorization of Technical Drawings](https://mlsanity.com/paper/2003.05471/) (2020)
- [ClipGen: A Deep Generative Model for Clipart Vectorization and Synthesis](https://mlsanity.com/paper/2106.04912/) (2021)
- [Real-Time Global Illumination Decomposition of Videos](https://mlsanity.com/paper/1908.01961/) (2019)
- [Inverse Diffusion Curves using Shape Optimization](https://mlsanity.com/paper/1610.02769/) (2016)
- [Effective Clipart Image Vectorization Through Direct Optimization of Bezigons](https://mlsanity.com/paper/1602.01913/) (2016)
- [SAMVG: A Multi-stage Image Vectorization Model with the Segment-Anything Model](https://mlsanity.com/paper/2311.05276/) (2023)

## Built on by (most cited first, 18 of 18)

- [Agent Banana: High-Fidelity Image Editing with Agentic Thinking and Tooling](https://mlsanity.com/paper/2602.09084/) (2026)
- [Render-in-the-Loop: Vector Graphics Generation via Visual Self-Feedback](https://mlsanity.com/paper/2604.20730/) (2026)
- [LayerPeeler: Autoregressive Peeling for Layer-wise Image Vectorization](https://mlsanity.com/paper/2505.23740/) (2025)
- [Bezier Splatting for Fast and Differentiable Vector Graphics Rendering](https://mlsanity.com/paper/2503.16424/) (2025)
- [Automatic Method Illustration Generation for AI Scientific Papers via Drawing Middleware Creation, Evolution, and Orchestration](https://mlsanity.com/paper/2603.29590/) (2026)
- [Image Vectorization with Depth: convexified shape layers with depth ordering](https://mlsanity.com/paper/2409.06648/) (2024)
- [AmodalSVG: Amodal Image Vectorization via Semantic Layer Peeling](https://mlsanity.com/paper/2604.10940/) (2026)
- [Vector Prism: Animating Vector Graphics by Stratifying Semantic Structure](https://mlsanity.com/paper/2512.14336/) (2025)
- [Empowering Vector Graphics with Consistently Arbitrary Viewing and View-dependent Visibility](https://mlsanity.com/paper/2505.21377/) (2025)
- [Birth of a Painting: Differentiable Brushstroke Reconstruction](https://mlsanity.com/paper/2511.13191/) (2025)
- [Controlling Your Image via Simplified Vector Graphics](https://mlsanity.com/paper/2602.14443/) (2026)
- [AnchorFlow: Learning Anchor Placement for Faithful and Editable SVG Reconstruction](https://mlsanity.com/paper/2605.19551/) (2026)
- [Clair Obscur: an Illumination-Aware Method for Real-World Image Vectorization](https://mlsanity.com/paper/2511.20034/) (2025)
- [SemLayer: Semantic-aware Generative Segmentation and Layer Construction for Abstract Icons](https://mlsanity.com/paper/2603.24039/) (2026)
- [Abstraction in Style: Beyond Texture and Color](https://mlsanity.com/paper/2603.29924/) (2026)
- [Vector Scaffolding: Inter-Scale Orchestration for Differentiable Image Vectorization](https://mlsanity.com/paper/2605.11913/) (2026)
- [NURBS Splatting: A Unified Differentiable Rendering Framework for Vector Graphics](https://mlsanity.com/paper/2606.31764/) (2026)
- [CubicSplat: Differentiable Vector Graphics via Error-Bounded Forward Relaxation](https://mlsanity.com/paper/2608.20803/) (2026)

## Abstract (arXiv)

This work presents a progressive image vectorization technique that reconstructs the raster image as layer-wise vectors from semantic-aligned macro structures to finer details. Our approach introduces a new image simplification method leveraging the feature-average effect in the Score Distillation Sampling mechanism, achieving effective visual abstraction from the detailed to coarse. Guided by the sequence of progressive simplified images, we propose a two-stage vectorization process of structural buildup and visual refinement, constructing the vectors in an organized and manageable manner. The resulting vectors are layered and well-aligned with the target image's explicit and implicit semantic structures. Our method demonstrates high performance across a wide range of images. Comparative analysis with existing vectorization methods highlights our technique's superiority in creating vectors with high visual fidelity, and more importantly, achieving higher semantic alignment and more compact layered representation. The project homepage is https://szuviz.github.io/layered_vectorization/.