mlsanityData to 13 Oct 2026

LoRACLR: Contrastive Adaptation for Customization of Diffusion Models

arXiv 2412.09622 ↗First posted 12 Dec 2024cs.CVcited by 20

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Recent advances in text-to-image customization have enabled high-fidelity, context-rich generation of personalized images, allowing specific concepts to appear in a variety of scenarios. However, current methods struggle with combining multiple personalized models, often leading to attribute entanglement or requiring separate training to preserve concept distinctiveness. We present LoRACLR, a novel approach for multi-concept image generation that merges multiple LoRA models, each fine-tuned for a distinct concept, into a single, unified model without additional individual fine-tuning. LoRACLR uses a contrastive objective to align and merge the weight spaces of these models, ensuring compatibility while minimizing interference. By enforcing distinct yet cohesive representations for each concept, LoRACLR enables efficient, scalable model composition for high-quality, multi-concept image synthesis. Our results highlight the effectiveness of LoRACLR in accurately merging multiple concepts, advancing the capabilities of personalized image generation.

Key facts. Cited by 20 papers in the map (data through 13 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 37 of its 43 references that are in the map. First posted on arXiv 12 Dec 2024.

Builds on

37 of 43 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Models16 Mar
  2. 2023Orthogonal Adaptation for Modular Customization of Diffusion Models5 Dec
  3. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
  4. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models29 May
  5. 2023Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Models5 Apr
  6. 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
  7. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr

Other references · 28

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,952
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,777
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,231
  5. 2020Denoising Diffusion Implicit Modelscited by 9,718
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,213
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,000
  8. 2022Classifier-Free Diffusion Guidancecited by 5,785
  9. 2022Generative Adversarial Networkscited by 5,614
  10. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,141
  11. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,751
  12. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,585
16 more
  1. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,619
  2. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,240
  3. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,289
  4. 2021Alias-Free Generative Adversarial Networkscited by 1,183
  5. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  6. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  7. 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 362
  8. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 344
  9. 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
  10. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 224
  11. 2023StyleDrop: Text-to-Image Generation in Any Stylecited by 199
  12. 2023ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAscited by 194
  13. 2021CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directionscited by 78
  14. 2021Multimodal Conditional Image Synthesis with Product-of-Experts GANscited by 75
  15. 2024OMG-Seg: Is One Model Good Enough For All Segmentation?cited by 74
  16. 2022Spatially-Adaptive Multilayer Selection for GAN Inversion and Editingcited by 39

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 11; 2026: 9.

20 papers.

Sort:

By area:

  1. 2025LoRAShop: Training-Free Multi-Concept Image Generation and Editing with Rectified Flow Transformerscited by 12
  2. 2025QR-LoRA: Efficient and Disentangled Fine-tuning via QR Decomposition for Customized Generationcited by 9
  3. 2025DyME: Dynamic Multi-Concept Erasure in Diffusion Models with Bi-Level Orthogonal LoRA Adaptationcited by 5
  4. 2025SplitFlux: Learning to Decouple Content and Style from a Single Imagecited by 5
  5. 2025LoRAtorio: An intrinsic approach to LoRA Skill Compositioncited by 3
  6. 2026Low-Rank Adaptation Redux for Large Modelscited by 3
  7. 2025JEDI: The Force of Jensen-Shannon Divergence in Disentangling Diffusion Modelscited by 1
  8. 2025Adversarial Concept Distillation for One-Step Diffusion Personalizationcited by 1
  9. 2026Generative Phomosaic with Structure-Aligned and Personalized Diffusioncited by 1
  10. 2026SeqLoRA: Bilevel Orthogonal Adaptation for Continual Multi-Concept Generationcited by 1
  11. 2026Conditioning noise is a free regularizer for LoRA fine-tuning: no pathology encoder required for diffusion-based artifact detection in histopathologycited by 1
  12. 2025Diffusion Sampling Path Tells More: An Efficient Plug-and-Play Strategy for Sample Filteringcited by 0
8 more
  1. 2025Deep Generative Continual Learning using Functional LoRA: FunLoRAcited by 0
  2. 2025FreeFuse: Multi-Subject LoRA Fusion via Adaptive Token-Level Routing at Test Timecited by 0
  3. 2025NP-LoRA: Null Space Projection for Subject-Style LoRA Fusioncited by 0
  4. 2026Training-Free Multi-Concept Image Editingcited by 0
  5. 2026PureCC: Pure Learning for Text-to-Image Concept Customizationcited by 0
  6. 2026Training-Free Multi-Concept LoRA Composition with Prompt-Aware Weightingcited by 0
  7. 2026CaricHarmony: Contrastive Diffusion Paths for Identity-Preserving Caricature Synthesiscited by 0
  8. 2026LoRango: It Takes Two LoRAs to Unlock Hidden Behaviors in Diffusion Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025TARA: Token-Aware LoRA for Composable Personalization in Diffusion Models
  2. 2024LoRA-Composer: Leveraging Low-Rank Adaptation for Multi-Concept Customization in Training-Free Diffusion Models
  3. 2025T-LoRA: Single Image Diffusion Model Customization Without Overfitting
  4. 2026LILAC: Layer-Wise Independent LoRAs and Cascaded Conditioning for Multi-Concept Customization of Diffusion Models
  5. 2024LoRA Diffusion: Zero-Shot LoRA Synthesis for Diffusion Model Personalization
  6. 2023Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluation
  7. 2023Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRA
  8. 2024Concept Weaver: Enabling Multi-Concept Fusion in Text-to-Image Models