mlsanityData to 14 Oct 2026

StructuredEdit: Constraint-Aware Graphic Design Editing via Differentiable Parameter Propagation

arXiv 2607.04612 ↗First posted 6 Jul 2026cs.GR cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

Abstract (arXiv)

Graphic design editing requires precise manipulation of typography, layout, and visual hierarchy under strict design constraints. Following the introduction of large language models, organizations have increasingly promoted vision-language models to enhance productivity. However, current models operate on pixels and achieve only 52% constraint satisfaction on structured design edits, thereby limiting their reliability for professional workflows. We present StructuredEdit, a pipeline that reframes design editing as parameter manipulation rather than pixel generation. Our core technical contribution is Differentiable Parameter Propagation (DPP), a training method that embeds hard design constraints into vision-language model fine-tuning by backpropagating pixel-level constraint violations through a lightweight differentiable rasterizer. A hybrid candidate-and-filter pipeline produces 125k validated edit triplets. The resulting system reaches 89% constraint satisfaction versus 52% for GPT-4V, 0.82 matched-element Intersection over Union, and 76% top-1 font accuracy over the 100 most-frequent design typefaces. In a user study (N=35), editing time drops 33% and correction iterations drop 44% relative to a GPT-4V baseline.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 4 of its 6 references that are in the map. First posted on arXiv 6 Jul 2026.

Builds on

4 of 6 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Texture or Semantics? Vision-Language Models Get Lost in Font Recognition31 Mar

Other references · 3

  1. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  2. 2021CanvasVAE: Learning to Generate Vector Graphic Documentscited by 95
  3. 2024Multimodal Markup Document Models for Graphic Design Completioncited by 16

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026InfoEdit: Probing Global Layout Reasoning in Infographic Editing
  2. 2026ReDesign: Recovering Editable Design Structures from Images via Agentic Decomposition
  3. 2026CreatiParser: Generative Image Parsing of Raster Graphic Designs into Editable Layers
  4. 2025PosterCopilot: Toward Layout Reasoning and Controllable Editing for Professional Graphic Design
  5. 2025SketchAssist: A Practical Assistant for Semantic Edits and Precise Local Redrawing
  6. 2026Graphic-Design-Bench: A Comprehensive Benchmark for Evaluating AI on Graphic Design Tasks
  7. 2026Can Vision Language Models Assess Graphic Design Aesthetics? A Benchmark, Evaluation, and Dataset Perspective
  8. 2026VCG-Bench: Towards A Unified Visual-Centric Benchmark for Structured Generation and Editing