# StyleAdapter: A Unified Stylized Image Generation Model

arXiv: https://arxiv.org/abs/2309.01770 · HTML: https://mlsanity.com/paper/2309.01770/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 48 references that are in the map.
- First posted on arXiv 4 Sep 2023.

## Builds on (most cited first, 30 of 40)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Generative Adversarial Text to Image Synthesis](https://mlsanity.com/paper/1605.05396/) (2016)
- [CogView: Mastering Text-to-Image Generation via Transformers](https://mlsanity.com/paper/2105.13290/) (2021)
- [AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks](https://mlsanity.com/paper/1711.10485/) (2017)
- [Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors](https://mlsanity.com/paper/2203.13131/) (2022)
- [Inversion-Based Style Transfer with Diffusion Models](https://mlsanity.com/paper/2211.13203/) (2022)

## Built on by (most cited first, 30 of 39)

- [CSGO: Content-Style Composition in Text-to-Image Generation](https://mlsanity.com/paper/2408.16766/) (2024)
- [StyleShot: A Snapshot on Any Style](https://mlsanity.com/paper/2407.01414/) (2024)
- [Efficient Diffusion Models: A Comprehensive Survey from Principles to Practices](https://mlsanity.com/paper/2410.11795/) (2024)
- [Attention Distillation: A Unified Approach to Visual Characteristics Transfer](https://mlsanity.com/paper/2502.20235/) (2025)
- [OmniConsistency: Learning Style-Agnostic Consistency from Paired Stylization Data](https://mlsanity.com/paper/2505.18445/) (2025)
- [A Survey on Personalized Content Synthesis with Diffusion Models](https://mlsanity.com/paper/2405.05538/) (2024)
- [Personalized Image Generation with Deep Generative Models: A Decade Survey](https://mlsanity.com/paper/2502.13081/) (2025)
- [Simulating the Visual World with Artificial Intelligence: A Roadmap](https://mlsanity.com/paper/2511.08585/) (2025)
- [VISTA: Triplet-Supervised Video Style Transfer with Diffusion Transformers](https://mlsanity.com/paper/2605.17312/) (2026)
- [An Efficient and Harmonized Framework for Balanced Cross-Domain Feature Integration](https://mlsanity.com/paper/2403.18461/) (2024)
- [CDST: Color Disentangled Style Transfer for Universal Style Reference Customization](https://mlsanity.com/paper/2506.13770/) (2025)
- [Lay2Story: Extending Diffusion Transformers for Layout-Togglable Story Generation](https://mlsanity.com/paper/2508.08949/) (2025)
- [TeleStyle: Content-Preserving Style Transfer in Images and Videos](https://mlsanity.com/paper/2601.20175/) (2026)
- [StyleAR: Customizing Multimodal Autoregressive Model for Style-Aligned Text-to-Image Generation](https://mlsanity.com/paper/2505.19874/) (2025)
- [QwenStyle: Content-Preserving Style Transfer with Qwen-Image-Edit](https://mlsanity.com/paper/2601.06202/) (2026)
- [Premier: Personalized Preference Modulation with Learnable User Embedding in Text-to-Image Generation](https://mlsanity.com/paper/2603.20725/) (2026)
- [MulSMo: Multimodal Stylized Motion Generation by Bidirectional Control Flow](https://mlsanity.com/paper/2412.09901/) (2024)
- [Only-Style: Stylistic Consistency in Image Generation without Content Leakage](https://mlsanity.com/paper/2506.09916/) (2025)
- [Jigsaw3D: Disentangled 3D Style Transfer via Patch Shuffling and Masking](https://mlsanity.com/paper/2510.10497/) (2025)
- [MoKus: Leveraging Cross-Modal Knowledge Transfer for Knowledge-Aware Concept Customization](https://mlsanity.com/paper/2603.12743/) (2026)
- [Style-CCL: Content-Preserving Style Transfer via Curriculum Continual Learning](https://mlsanity.com/paper/2606.14746/) (2026)
- [DreamStyle3D: Efficient 3D Stylized Asset Generation via Dual-Attention Disentanglement](https://mlsanity.com/paper/2607.24721/) (2026)
- [ArtWeaver: Advanced Dynamic Style Integration via Diffusion Model](https://mlsanity.com/paper/2405.15287/) (2024)
- [Less is More: Masking Elements in Image Condition Features Avoids Content Leakages in Style Transfer Diffusion Models](https://mlsanity.com/paper/2502.07466/) (2025)
- [ICAS: IP Adapter and ControlNet-based Attention Structure for Multi-Subject Style Transfer Optimization](https://mlsanity.com/paper/2504.13224/) (2025)
- [SPG: Style-Prompting Guidance for Style-Specific Content Creation](https://mlsanity.com/paper/2508.11476/) (2025)
- [DUDE: Diffusion-Based Unsupervised Cross-Domain Image Retrieval](https://mlsanity.com/paper/2509.04193/) (2025)
- [SSGaussian: Semantic-Aware and Structure-Preserving 3D Style Transfer](https://mlsanity.com/paper/2509.04379/) (2025)
- [PaintFlow: A Unified Framework for Interactive Oil Paintings Editing and Generation](https://mlsanity.com/paper/2512.08534/) (2025)
- [Sissi: Zero-shot Style-guided Image Synthesis via Semantic-style Integration](https://mlsanity.com/paper/2601.06605/) (2026)

## Abstract (arXiv)

This work focuses on generating high-quality images with specific style of reference images and content of provided textual descriptions. Current leading algorithms, i.e., DreamBooth and LoRA, require fine-tuning for each style, leading to time-consuming and computationally expensive processes. In this work, we propose StyleAdapter, a unified stylized image generation model capable of producing a variety of stylized images that match both the content of a given prompt and the style of reference images, without the need for per-style fine-tuning. It introduces a two-path cross-attention (TPCA) module to separately process style information and textual prompt, which cooperate with a semantic suppressing vision model (SSVM) to suppress the semantic content of style images. In this way, it can ensure that the prompt maintains control over the content of the generated images, while also mitigating the negative impact of semantic information in style references. This results in the content of the generated image adhering to the prompt, and its style aligning with the style references. Besides, our StyleAdapter can be integrated with existing controllable synthesis methods, such as T2I-adapter and ControlNet, to attain a more controllable and stable generation process. Extensive experiments demonstrate the superiority of our method over previous works.