StructuredEdit: Constraint-Aware Graphic Design Editing via Differentiable Parameter Propagation
On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG
Abstract (arXiv)
Graphic design editing requires precise manipulation of typography, layout, and visual hierarchy under strict design constraints. Following the introduction of large language models, organizations have increasingly promoted vision-language models to enhance productivity. However, current models operate on pixels and achieve only 52% constraint satisfaction on structured design edits, thereby limiting their reliability for professional workflows. We present StructuredEdit, a pipeline that reframes design editing as parameter manipulation rather than pixel generation. Our core technical contribution is Differentiable Parameter Propagation (DPP), a training method that embeds hard design constraints into vision-language model fine-tuning by backpropagating pixel-level constraint violations through a lightweight differentiable rasterizer. A hybrid candidate-and-filter pipeline produces 125k validated edit triplets. The resulting system reaches 89% constraint satisfaction versus 52% for GPT-4V, 0.82 matched-element Intersection over Union, and 76% top-1 font accuracy over the 100 most-frequent design typefaces. In a user study (N=35), editing time drops 33% and correction iterations drop 44% relative to a GPT-4V baseline.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 4 of its 6 references that are in the map. First posted on arXiv 6 Jul 2026.
Builds on
4 of 6 references.
Closest ancestors · 1
Same direction, newest first
Other references · 3
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
- 2021CanvasVAE: Learning to Generate Vector Graphic Documentscited by 95
- 2024Multimodal Markup Document Models for Graphic Design Completioncited by 16
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026InfoEdit: Probing Global Layout Reasoning in Infographic Editing
- 2026ReDesign: Recovering Editable Design Structures from Images via Agentic Decomposition
- 2026CreatiParser: Generative Image Parsing of Raster Graphic Designs into Editable Layers
- 2025PosterCopilot: Toward Layout Reasoning and Controllable Editing for Professional Graphic Design
- 2025SketchAssist: A Practical Assistant for Semantic Edits and Precise Local Redrawing
- 2026Graphic-Design-Bench: A Comprehensive Benchmark for Evaluating AI on Graphic Design Tasks
- 2026Can Vision Language Models Assess Graphic Design Aesthetics? A Benchmark, Evaluation, and Dataset Perspective
- 2026VCG-Bench: Towards A Unified Visual-Centric Benchmark for Structured Generation and Editing