# Multi-LoRA Composition for Image Generation

arXiv: https://arxiv.org/abs/2402.16843 · HTML: https://mlsanity.com/paper/2402.16843/

## Key facts

- Cited by 80 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 41 references that are in the map.
- First posted on arXiv 26 Feb 2024.

## Builds on (most cited first, 30 of 34)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning](https://mlsanity.com/paper/2012.13255/) (2020)
- [Compositional Visual Generation with Composable Diffusion Models](https://mlsanity.com/paper/2206.01714/) (2022)
- [Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors](https://mlsanity.com/paper/2203.13131/) (2022)
- [Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis](https://mlsanity.com/paper/2212.05032/) (2022)
- [LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition](https://mlsanity.com/paper/2307.13269/) (2023)
- [Image Generation from Scene Graphs](https://mlsanity.com/paper/1804.01622/) (2018)
- [Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC](https://mlsanity.com/paper/2302.11552/) (2023)
- [StyleDrop: Text-to-Image Generation in Any Style](https://mlsanity.com/paper/2306.00983/) (2023)
- [ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAs](https://mlsanity.com/paper/2311.13600/) (2023)
- [Composing Parameter-Efficient Modules with Arithmetic Operations](https://mlsanity.com/paper/2306.14870/) (2023)
- [GPT-4V(ision) as a Generalist Evaluator for Vision-Language Tasks](https://mlsanity.com/paper/2311.01361/) (2023)
- [Collaborative Diffusion for Multi-Modal Face Generation and Editing](https://mlsanity.com/paper/2304.10530/) (2023)
- [In-Context Learning Unlocked for Diffusion Models](https://mlsanity.com/paper/2305.01115/) (2023)
- [ImagenHub: Standardizing the evaluation of conditional image generation models](https://mlsanity.com/paper/2310.01596/) (2023)
- [Learning to Compose Visual Relations](https://mlsanity.com/paper/2111.09297/) (2021)
- [Controllable and Compositional Generation with Latent-Space Energy-Based Models](https://mlsanity.com/paper/2110.10873/) (2021)
- [Modeling Image Composition for Complex Scene Generation](https://mlsanity.com/paper/2206.00923/) (2022)

## Built on by (most cited first, 30 of 80)

- [ReFT: Representation Finetuning for Language Models](https://mlsanity.com/paper/2404.03592/) (2024)
- [LoRA-Composer: Leveraging Low-Rank Adaptation for Multi-Concept Customization in Training-Free Diffusion Models](https://mlsanity.com/paper/2403.11627/) (2024)
- [LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks](https://mlsanity.com/paper/2410.13025/) (2024)
- [K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAs](https://mlsanity.com/paper/2502.18461/) (2025)
- [HiCo: Hierarchical Controllable Diffusion Model for Layout-to-image Generation](https://mlsanity.com/paper/2410.14324/) (2024)
- [Low-Rank Adaptation for Foundation Models: A Comprehensive Review](https://mlsanity.com/paper/2501.00365/) (2024)
- [The Superposition of Diffusion Models Using the Ito Density Estimator](https://mlsanity.com/paper/2412.17762/) (2024)
- [How to Continually Adapt Text-to-Image Diffusion Models for Flexible Customization?](https://mlsanity.com/paper/2410.17594/) (2024)
- [Getting it Right: Improving Spatial Consistency in Text-to-Image Models](https://mlsanity.com/paper/2404.01197/) (2024)
- [MC2: Multi-concept Guidance for Customized Multi-concept Generation](https://mlsanity.com/paper/2404.05268/) (2024)
- [AlignGuard: Scalable Safety Alignment for Text-to-Image Generation](https://mlsanity.com/paper/2412.10493/) (2024)
- [FouRA: Fourier Low Rank Adaptation](https://mlsanity.com/paper/2406.08798/) (2024)
- [SELMA: Learning and Merging Skill-Specific Text-to-Image Experts with Auto-Generated Data](https://mlsanity.com/paper/2403.06952/) (2024)
- [Retrieval-Augmented Mixture of LoRA Experts for Uploadable Machine Learning](https://mlsanity.com/paper/2406.16989/) (2024)
- [Skill Expansion and Composition in Parameter Space](https://mlsanity.com/paper/2502.05932/) (2025)
- [Realistic Evaluation of Model Merging for Compositional Generalization](https://mlsanity.com/paper/2409.18314/) (2024)
- [IterIS: Iterative Inference-Solving Alignment for LoRA Merging](https://mlsanity.com/paper/2411.15231/) (2024)
- [Contrastive Test-Time Composition of Multiple LoRA Models for Image Generation](https://mlsanity.com/paper/2403.19776/) (2024)
- [3-in-1: 2D Rotary Adaptation for Efficient Finetuning, Efficient Batching and Composability](https://mlsanity.com/paper/2409.00119/) (2024)
- [S2FT: Efficient, Scalable and Generalizable LLM Fine-tuning by Structured Sparsity](https://mlsanity.com/paper/2412.06289/) (2024)
- [Towards Higher Effective Rank in Parameter-efficient Fine-tuning using Khatri--Rao Product](https://mlsanity.com/paper/2508.00230/) (2025)
- [EigenLoRAx: Recycling Adapters to Find Principal Subspaces for Resource-Efficient Adaptation and Inference](https://mlsanity.com/paper/2502.04700/) (2025)
- [QR-LoRA: Efficient and Disentangled Fine-tuning via QR Decomposition for Customized Generation](https://mlsanity.com/paper/2507.04599/) (2025)
- [Enhancing Adversarial Robustness of Vision-Language Models through Low-Rank Adaptation](https://mlsanity.com/paper/2404.13425/) (2024)
- [DreamRunner: Fine-Grained Compositional Story-to-Video Generation with Retrieval-Augmented Motion Adaptation](https://mlsanity.com/paper/2411.16657/) (2024)
- [Diffusion Blend: Inference-Time Multi-Preference Alignment for Diffusion Models](https://mlsanity.com/paper/2505.18547/) (2025)
- [LiON-LoRA: Rethinking LoRA Fusion to Unify Controllable Spatial and Temporal Generation for Video Diffusion](https://mlsanity.com/paper/2507.05678/) (2025)
- [Training-Free Multi-Style Fusion Through Reference-Based Adaptive Modulation](https://mlsanity.com/paper/2509.18602/) (2025)
- [Can video generation replace cinematographers? Research on the cinematic language of generated video](https://mlsanity.com/paper/2412.12223/) (2024)
- [Fine Tuning without Catastrophic Forgetting via Selective Low Rank Adaptation](https://mlsanity.com/paper/2501.15377/) (2025)

## Abstract (arXiv)

Low-Rank Adaptation (LoRA) is extensively utilized in text-to-image models for the accurate rendition of specific elements like distinct characters or unique styles in generated images. Nonetheless, existing methods face challenges in effectively composing multiple LoRAs, especially as the number of LoRAs to be integrated grows, thus hindering the creation of complex imagery. In this paper, we study multi-LoRA composition through a decoding-centric perspective. We present two training-free methods: LoRA Switch, which alternates between different LoRAs at each denoising step, and LoRA Composite, which simultaneously incorporates all LoRAs to guide more cohesive image synthesis. To evaluate the proposed approaches, we establish ComposLoRA, a new comprehensive testbed as part of this research. It features a diverse range of LoRA categories with 480 composition sets. Utilizing an evaluation framework based on GPT-4V, our findings demonstrate a clear improvement in performance with our methods over the prevalent baseline, particularly evident when increasing the number of LoRAs in a composition. The code, benchmarks, LoRA weights, and all evaluation details are available on our project website: https://maszhongming.github.io/Multi-LoRA-Composition.