SketchAssist: A Practical Assistant for Semantic Edits and Precise Local Redrawing
Not on the map. Why?
Abstract (arXiv)
Sketch editing requires jointly handling high-level semantic changes and precise local redrawing, a combination that is particularly challenging for sparse, style-sensitive line art. Unlike natural images, sketches rely on minimal visual cues, making it difficult for existing methods to reconcile global semantic modifications with fine-grained structural control while preserving overall coherence. We present SketchAssist, an interactive sketch assistant that unifies instruction-guided editing with line-guided region redrawing, enabling efficient and controllable sketch manipulation while preserving overall composition. To support this task, we introduce a controllable data generation pipeline that constructs structured edit sequences with precise attribute variations and maintains structural alignment across multi-step modifications, while expanding stylistic diversity via style-preserving transformations. Building on this data, SketchAssist adopts a unified framework based on DiT, using a multi-channel input representation to encode sketches, masks, and guidance signals within a single interface. To further handle different editing modes, we integrate a Task-guided Mixture-of-Experts (T-MoE) into LoRA layers, enabling adaptive control over semantic and structural guidance. Extensive experiments demonstrate state-of-the-art performance on both tasks, achieving strong instruction adherence and improved structural and style consistency compared to recent methods. Together, our method provide a practical and controllable solution for sketch editing.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 45 references that are in the map. First posted on arXiv 16 Dec 2025.
Builds on
38 of 45 references.
References · 38
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
26 more
- 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
- 2021Alias-Free Generative Adversarial Networkscited by 1,184
- 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,178
- 2025Qwen-Image Technical Reportcited by 1,044
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
- 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 927
- 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
- 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 340
- 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 324
- 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
- 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
- 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 241
- 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 204
- 2024BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusioncited by 202
- 2023VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluationcited by 201
- 2023Human-Art: A Versatile Human-Centric Dataset Bridging Natural and Artificial Scenescited by 68
- 2021SketchEdit: Mask-Free Local Image Manipulation with Partial Sketchescited by 33
- 2024MagicQuill: An Intelligent Interactive Image Editing Systemcited by 30
- 2024Training-free Regional Prompting for Diffusion Transformerscited by 27
- 2024SketchINR: A First Look into Sketches as Implicit Neural Representationscited by 13
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026SI-Edit: Toward Sketch-Instruction Guided Local Image Editing with Pixel-Level Precision
- 2026Component-Aware Sketch-to-Image Generation Using Self-Attention Encoding and Coordinate-Preserving Fusion
- 2025ReMix: Towards a Unified View of Consistent Character Generation and Editing
- 2026CalliMaster: Mastering Page-level Chinese Calligraphy via Layout-guided Spatial Planning
- 2025Text to Sketch Generation with Multi-Styles
- 2026StructuredEdit: Constraint-Aware Graphic Design Editing via Differentiable Parameter Propagation
- 2025Neural Scene Designer: Self-Styled Semantic Image Manipulation
- 2026Editing on the Generative Manifold: A Theoretical and Empirical Study of General Diffusion-Based Image Editing Trade-offs