Text to Sketch Generation with Multi-Styles
On the map: Multimodal LLMs / vision-language models → Image editing and quality assessment
Abstract (arXiv)
Recent advances in vision-language models have facilitated progress in sketch generation. However, existing specialized methods primarily focus on generic synthesis and lack mechanisms for precise control over sketch styles. In this work, we propose a training-free framework based on diffusion models that enables explicit style guidance via textual prompts and referenced style sketches. Unlike previous style transfer methods that overwrite key and value matrices in self-attention, we incorporate the reference features as auxiliary information with linear smoothing and leverage a style-content guidance mechanism. This design effectively reduces content leakage from reference sketches and enhances synthesis quality, especially in cases with low structural similarity between reference and target sketches. Furthermore, we extend our framework to support controllable multi-style generation by integrating features from multiple reference sketches, coordinated via a joint AdaIN module. Extensive experiments demonstrate that our approach achieves high-quality sketch generation with accurate style alignment and improved flexibility in style control. The official implementation of M3S is available at https://github.com/CMACH508/M3S.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 44 of its 63 references that are in the map. First posted on arXiv 6 Nov 2025.
Builds on
44 of 63 references.
References · 44
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
- 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
32 more
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
- 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
- 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
- 2017Universal Style Transfer via Feature Transformscited by 354
- 2017A Neural Representation of Sketch Drawingscited by 313
- 2022Inversion-Based Style Transfer with Diffusion Modelscited by 291
- 2023An Edit Friendly DDPM Noise Space: Inversion and Manipulationscited by 244
- 2023Style Aligned Image Generation via Shared Attentioncited by 216
- 2017Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesiscited by 215
- 2023Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfercited by 196
- 2023Cross-Image Attention for Zero-Shot Appearance Transfercited by 136
- 2024Implicit Style-Content Separation using B-LoRAcited by 135
- 2021Noise2Score: Tweedie's Approach to Self-Supervised Image Denoising without Clean Imagescited by 113
- 2022VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Modelscited by 112
- 2024CSGO: Content-Style Composition in Text-to-Image Generationcited by 78
- 2019Photo-Sketching: Inferring Contour Drawings from Imagescited by 68
- 2024DEADiff: An Efficient Stylization Diffusion Model with Disentangled Representationscited by 68
- 2022CLIPascene: Scene Sketching with Different Types and Levels of Abstractioncited by 64
- 2020Deep Learning for Free-Hand Sketch: A Surveycited by 59
- 2023DiffSketcher: Text Guided Vector Sketch Synthesis through Latent Diffusion Modelscited by 54
- 2024RB-Modulation: Training-Free Personalization of Diffusion Models using Stochastic Optimal Controlcited by 48
- 2018Learning to Sketch with Shortcut Cycle Consistencycited by 47
- 2025Attention Distillation: A Unified Approach to Visual Characteristics Transfercited by 33
- 2024SketchAgent: Language-Driven Sequential Sketch Generationcited by 31
- 2024StyleStudio: Text-Driven Style Transfer with Selective Control of Style Elementscited by 26
- 2017Sketch-pix2seq: a Model to Generate Sketches of Multiple Categoriescited by 20
- 2021SketchLattice: Latticed Representation for Sketch Manipulationcited by 19
- 2023ChiroDiff: Modelling chirographic data with Diffusion Modelscited by 14
- 2023SEVA: Leveraging sketches to evaluate alignment between human and machine visual abstractioncited by 14
- 2024Semi-supervised reference-based sketch extraction using a contrastive learning frameworkcited by 5
- 2022Linking Sketch Patches by Learning Synonymous Proximity for Graphic Sketch Representationcited by 4
- 2025MixSA: Training-free Reference-based Sketch Extraction via Mixture-of-Self-Attentioncited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Training-Free Sketch-Guided Diffusion with Latent Optimization
- 2024Visual Style Prompting with Swapping Self-Attention
- 2024Sketch2NeRF: Multi-view Sketch-guided Text-to-3D Generation
- 2023Reference-based Image Composition with Sketch via Structure-aware Diffusion Model
- 2025AutoSketch: VLM-assisted Style-Aware Vector Sketch Completion
- 2022Arbitrary Style Guidance for Enhanced Diffusion-Based Text-to-Image Generation
- 2022Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Models
- 2024SketchDream: Sketch-based Text-to-3D Generation and Editing