Sissi: Zero-shot Style-guided Image Synthesis via Semantic-style Integration
On the map: Diffusion models → Image generation, text-to-image and editing
Abstract (arXiv)
Text-guided image generation has advanced rapidly with large-scale diffusion models, yet achieving precise stylization with visual exemplars remains difficult. Existing approaches often depend on task-specific retraining or expensive inversion procedures, which can compromise content integrity, reduce style fidelity, and lead to an unsatisfactory trade-off between semantic prompt adherence and style alignment. In this work, we introduce a training-free framework that reformulates style-guided synthesis as an in-context learning task. Guided by textual semantic prompts, our method concatenates a reference style image with a masked target image, leveraging a pretrained ReFlow-based inpainting model to seamlessly integrate semantic content with the desired style through multimodal attention fusion. We further analyze the imbalance and noise sensitivity inherent in multimodal attention fusion and propose a Dynamic Semantic-Style Integration (DSSI) mechanism that reweights attention between textual semantic and style visual tokens, effectively resolving guidance conflicts and enhancing output coherence. Experiments show that our approach achieves high-fidelity stylization with superior semantic-style balance and visual quality, offering a simple yet powerful alternative to complex, artifact-prone prior methods.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 47 of its 56 references that are in the map. First posted on arXiv 10 Jan 2026.
Builds on
47 of 56 references.
Closest ancestors · 16
Same direction, newest first
- 2025T-LoRA: Single Image Diffusion Model Customization Without Overfitting8 Jul
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space17 Jun
- 2024FireFlow: Fast Inversion of Rectified Flow for Image Semantic Editing10 Dec
- 2024In-Context LoRA for Diffusion Transformers31 Oct
- 2024CSGO: Content-Style Composition in Text-to-Image Generation29 Aug
- 2024DEADiff: An Efficient Stylization Diffusion Model with Disentangled Representations11 Mar
- 2024Visual Style Prompting with Swapping Self-Attention20 Feb
- 2023Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfer11 Dec
- 2023InstaStyle: Inversion Noise of a Stylized Image is Secretly a Style Adviser25 Nov
- 2023Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRA12 Apr
- 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
- 2022Inversion-Based Style Transfer with Diffusion Models23 Nov
4 more
- 2022DreamArtist++: Controllable One-Shot Text-to-Image Generation via Positive-Negative Adapter21 Nov
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
Other references · 31
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
- 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
- 2023Exploiting Diffusion Prior for Real-World Image Super-Resolutioncited by 489
- 2017Universal Style Transfer via Feature Transformscited by 354
- 2022Images Speak in Images: A Generalist Painter for In-Context Visual Learningcited by 318
- 2023SinSR: Diffusion-Based Image Super-Resolution in a Single Stepcited by 238
19 more
- 2023Style Aligned Image Generation via Shared Attentioncited by 216
- 2018Arbitrary Style Transfer with Style-Attentional Networkscited by 198
- 2016Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networkscited by 192
- 2021StyTr2: Image Style Transfer with Transformerscited by 187
- 2024InstantStyle: Free Lunch towards Style-Preserving in Text-to-Image Generationcited by 170
- 2017Demystifying Neural Style Transfercited by 152
- 2021ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flowscited by 113
- 2022Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learningcited by 93
- 2024StyleShot: A Snapshot on Any Stylecited by 62
- 2024Large-Scale Text-to-Image Model with Inpainting is a Zero-Shot Subject-Driven Image Generatorcited by 48
- 2023ArtAdapter: Text-to-Image Style Transfer using Multi-Level Style Encoder and Explicit Adaptationcited by 40
- 2023StyleAdapter: A Unified Stylized Image Generation Modelcited by 39
- 2023General Image-to-Image Translation with One-Shot Image Guidancecited by 31
- 2023Context Diffusion: In-Context Aware Image Generationcited by 21
- 2025Multi-turn Consistent Image Editingcited by 18
- 2025In-Context Brush: Zero-shot Customized Subject Insertion with Context-Aware Latent Space Manipulationcited by 7
- 2025SaMam: Style-aware State Space Model for Arbitrary Image Style Transfercited by 5
- 2024Z-STAR+: A Zero-shot Style Transfer Method via Adjusting Style Distributioncited by 4
- 2025Z-Magic: Zero-shot Multiple Attributes Guided Image Creatorcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024ArtWeaver: Advanced Dynamic Style Integration via Diffusion Model
- 2025StyleBlend: Enhancing Style-Specific Content Creation in Text-to-Image Diffusion Models
- 2023PARASOL: Parametric Style Control for Diffusion Image Synthesis
- 2024Beyond Color and Lines: Zero-Shot Style-Specific Image Variations with Coordinated Semantics
- 2025Neural Scene Designer: Self-Styled Semantic Image Manipulation
- 2026CoCoDiff: Correspondence-Consistent Diffusion Model for Fine-grained Style Transfer
- 2024StyleMamba : State Space Model for Efficient Text-driven Image Style Transfer
- 2026Injecting Image Guidance into Text-Conditioned Diffusion Models at Inference