mlsanityData to 14 Oct 2026

Text to Sketch Generation with Multi-Styles

arXiv 2511.04123 ↗First posted 6 Nov 2025cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Image editing and quality assessment

Abstract (arXiv)

Recent advances in vision-language models have facilitated progress in sketch generation. However, existing specialized methods primarily focus on generic synthesis and lack mechanisms for precise control over sketch styles. In this work, we propose a training-free framework based on diffusion models that enables explicit style guidance via textual prompts and referenced style sketches. Unlike previous style transfer methods that overwrite key and value matrices in self-attention, we incorporate the reference features as auxiliary information with linear smoothing and leverage a style-content guidance mechanism. This design effectively reduces content leakage from reference sketches and enhances synthesis quality, especially in cases with low structural similarity between reference and target sketches. Furthermore, we extend our framework to support controllable multi-style generation by integrating features from multiple reference sketches, coordinated via a joint AdaIN module. Extensive experiments demonstrate that our approach achieves high-quality sketch generation with accurate style alignment and improved flexibility in style control. The official implementation of M3S is available at https://github.com/CMACH508/M3S.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 44 of its 63 references that are in the map. First posted on arXiv 6 Nov 2025.

Builds on

44 of 63 references.

References · 44

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  8. 2022Classifier-Free Diffusion Guidancecited by 5,847
  9. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  11. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  12. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
32 more
  1. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  2. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  3. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
  4. 2017Universal Style Transfer via Feature Transformscited by 354
  5. 2017A Neural Representation of Sketch Drawingscited by 313
  6. 2022Inversion-Based Style Transfer with Diffusion Modelscited by 291
  7. 2023An Edit Friendly DDPM Noise Space: Inversion and Manipulationscited by 244
  8. 2023Style Aligned Image Generation via Shared Attentioncited by 216
  9. 2017Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesiscited by 215
  10. 2023Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfercited by 196
  11. 2023Cross-Image Attention for Zero-Shot Appearance Transfercited by 136
  12. 2024Implicit Style-Content Separation using B-LoRAcited by 135
  13. 2021Noise2Score: Tweedie's Approach to Self-Supervised Image Denoising without Clean Imagescited by 113
  14. 2022VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Modelscited by 112
  15. 2024CSGO: Content-Style Composition in Text-to-Image Generationcited by 78
  16. 2019Photo-Sketching: Inferring Contour Drawings from Imagescited by 68
  17. 2024DEADiff: An Efficient Stylization Diffusion Model with Disentangled Representationscited by 68
  18. 2022CLIPascene: Scene Sketching with Different Types and Levels of Abstractioncited by 64
  19. 2020Deep Learning for Free-Hand Sketch: A Surveycited by 59
  20. 2023DiffSketcher: Text Guided Vector Sketch Synthesis through Latent Diffusion Modelscited by 54
  21. 2024RB-Modulation: Training-Free Personalization of Diffusion Models using Stochastic Optimal Controlcited by 48
  22. 2018Learning to Sketch with Shortcut Cycle Consistencycited by 47
  23. 2025Attention Distillation: A Unified Approach to Visual Characteristics Transfercited by 33
  24. 2024SketchAgent: Language-Driven Sequential Sketch Generationcited by 31
  25. 2024StyleStudio: Text-Driven Style Transfer with Selective Control of Style Elementscited by 26
  26. 2017Sketch-pix2seq: a Model to Generate Sketches of Multiple Categoriescited by 20
  27. 2021SketchLattice: Latticed Representation for Sketch Manipulationcited by 19
  28. 2023ChiroDiff: Modelling chirographic data with Diffusion Modelscited by 14
  29. 2023SEVA: Leveraging sketches to evaluate alignment between human and machine visual abstractioncited by 14
  30. 2024Semi-supervised reference-based sketch extraction using a contrastive learning frameworkcited by 5
  31. 2022Linking Sketch Patches by Learning Synonymous Proximity for Graphic Sketch Representationcited by 4
  32. 2025MixSA: Training-free Reference-based Sketch Extraction via Mixture-of-Self-Attentioncited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Training-Free Sketch-Guided Diffusion with Latent Optimization
  2. 2024Visual Style Prompting with Swapping Self-Attention
  3. 2024Sketch2NeRF: Multi-view Sketch-guided Text-to-3D Generation
  4. 2023Reference-based Image Composition with Sketch via Structure-aware Diffusion Model
  5. 2025AutoSketch: VLM-assisted Style-Aware Vector Sketch Completion
  6. 2022Arbitrary Style Guidance for Enhanced Diffusion-Based Text-to-Image Generation
  7. 2022Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Models
  8. 2024SketchDream: Sketch-based Text-to-3D Generation and Editing