# StyleStudio: Text-Driven Style Transfer with Selective Control of Style Elements

arXiv: https://arxiv.org/abs/2412.08503 · HTML: https://mlsanity.com/paper/2412.08503/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 42 references that are in the map.
- First posted on arXiv 11 Dec 2024.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation](https://mlsanity.com/paper/2211.12572/) (2022)
- [CogView: Mastering Text-to-Image Generation via Transformers](https://mlsanity.com/paper/2105.13290/) (2021)
- [Diffusion Models already have a Semantic Latent Space](https://mlsanity.com/paper/2210.10960/) (2022)
- [Inversion-Based Style Transfer with Diffusion Models](https://mlsanity.com/paper/2211.13203/) (2022)
- [Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystack](https://mlsanity.com/paper/2309.15807/) (2023)
- [Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfer](https://mlsanity.com/paper/2312.09008/) (2023)
- [InstantStyle: Free Lunch towards Style-Preserving in Text-to-Image Generation](https://mlsanity.com/paper/2404.02733/) (2024)
- [Towards Understanding Cross and Self-Attention in Stable Diffusion for Text-Guided Image Editing](https://mlsanity.com/paper/2403.03431/) (2024)
- [Implicit Style-Content Separation using B-LoRA](https://mlsanity.com/paper/2403.14572/) (2024)
- [CSGO: Content-Style Composition in Text-to-Image Generation](https://mlsanity.com/paper/2408.16766/) (2024)
- [Visual Style Prompting with Swapping Self-Attention](https://mlsanity.com/paper/2402.12974/) (2024)
- [DreamMatcher: Appearance Matching Self-Attention for Semantically-Consistent Text-to-Image Personalization](https://mlsanity.com/paper/2402.09812/) (2024)
- [FreeCustom: Tuning-Free Customized Image Generation for Multi-Concept Composition](https://mlsanity.com/paper/2405.13870/) (2024)

## Built on by (most cited first, 26 of 26)

- [USO: Unified Style and Subject-Driven Generation via Disentangled and Reward Learning](https://mlsanity.com/paper/2508.18966/) (2025)
- [Auto-Slides: An Interactive Multi-Agent System for Creating and Customizing Research Presentations](https://mlsanity.com/paper/2509.11062/) (2025)
- [Distilling Parallel Gradients for Fast ODE Solvers of Diffusion Models](https://mlsanity.com/paper/2507.14797/) (2025)
- [QR-LoRA: Efficient and Disentangled Fine-tuning via QR Decomposition for Customized Generation](https://mlsanity.com/paper/2507.04599/) (2025)
- [Adaptive Stochastic Coefficients for Accelerating Diffusion Sampling](https://mlsanity.com/paper/2510.23285/) (2025)
- [StyleTextGen: Style-Conditioned Multilingual Scene Text Generation](https://mlsanity.com/paper/2605.14708/) (2026)
- [Detail++: Training-Free Detail Enhancer for T2I Diffusion Models](https://mlsanity.com/paper/2507.17853/) (2025)
- [Loom: Diffusion-Transformer for Interleaved Generation](https://mlsanity.com/paper/2512.18254/) (2025)
- [TP-Blend: Textual-Prompt Attention Pairing for Precise Object-Style Blending in Diffusion Models](https://mlsanity.com/paper/2601.08011/) (2026)
- [Parallel Diffusion Solver via Residual Dirichlet Policy Optimization](https://mlsanity.com/paper/2512.22796/) (2025)
- [CleanStyle: Plug-and-Play Style Conditioning Purification for Text-to-Image Stylization](https://mlsanity.com/paper/2602.20721/) (2026)
- [Budget-Constrained Step-Level Diffusion Caching](https://mlsanity.com/paper/2606.13496/) (2026)
- [TransLight: Image-Guided Customized Lighting Control with Generative Decoupling](https://mlsanity.com/paper/2508.14814/) (2025)
- [CoDi: Subject-Consistent and Pose-Diverse Text-to-Image Generation](https://mlsanity.com/paper/2507.08396/) (2025)
- [A Style is Worth One Code: Unlocking Code-to-Style Image Generation with Discrete Style Space](https://mlsanity.com/paper/2511.10555/) (2025)
- [RegionRoute: Regional Style Transfer with Diffusion Model](https://mlsanity.com/paper/2602.19254/) (2026)
- [Recolour What Matters: Region-Aware Colour Editing via Token-Level Diffusion](https://mlsanity.com/paper/2603.18466/) (2026)
- [CameraEditor: Camera-Controlled Image Editing via Video-Prior Sequential Modeling](https://mlsanity.com/paper/2609.01479/) (2026)
- [Text to Sketch Generation with Multi-Styles](https://mlsanity.com/paper/2511.04123/) (2025)
- [Style-Aware Gloss Control for Generative Non-Photorealistic Rendering](https://mlsanity.com/paper/2602.16611/) (2026)
- [UniCSG: Unified High-Fidelity Content-Constrained Style-Driven Generation via Staged Semantic and Frequency Disentanglement](https://mlsanity.com/paper/2604.17850/) (2026)
- [Structured 3D Latents Are Surprisingly Powerful: Unleashing Generalizable Style with 2D Diffusion](https://mlsanity.com/paper/2605.04412/) (2026)
- [DICT: Data Injection and Contrastive Trajectory Refinement for Conditional Image Generation with Diffusion Models](https://mlsanity.com/paper/2607.03899/) (2026)
- [Rare Concept Generation via Counterfactual Inference in Diffusion Models](https://mlsanity.com/paper/2607.14765/) (2026)
- [SafeStyle: Calibrated Style Residual Injection for Controllable Style-Leakage Trade-off in Diffusion Stylization](https://mlsanity.com/paper/2609.21242/) (2026)
- [Beyond Emotion Prompts: Fine-Grained Text-to-Image Generation Driven by Valence-Arousal-Dominance](https://mlsanity.com/paper/2609.24215/) (2026)

## Abstract (arXiv)

Text-driven style transfer aims to merge the style of a reference image with content described by a text prompt. Recent advancements in text-to-image models have improved the nuance of style transformations, yet significant challenges remain, particularly with overfitting to reference styles, limiting stylistic control, and misaligning with textual content. In this paper, we propose three complementary strategies to address these issues. First, we introduce a cross-modal Adaptive Instance Normalization (AdaIN) mechanism for better integration of style and text features, enhancing alignment. Second, we develop a Style-based Classifier-Free Guidance (SCFG) approach that enables selective control over stylistic elements, reducing irrelevant influences. Finally, we incorporate a teacher model during early generation stages to stabilize spatial layouts and mitigate artifacts. Our extensive evaluations demonstrate significant improvements in style transfer quality and alignment with textual prompts. Furthermore, our approach can be integrated into existing style transfer frameworks without fine-tuning.