SSR-Merge: Subspace Signal Routing for Training-Free LoRA Merging in Diffusion Models
On the map: Diffusion models → Diffusion foundations and sampling
Abstract (arXiv)
Low-Rank Adaptation (LoRA) merging can efficiently combine diverse generative capabilities from multiple trained LoRAs for a diffusion model. However, existing LoRA merging techniques often suffer from severe parameter interference, causing destructive collisions in the shared parameter space. To address this, we propose Subspace Signal Routing (SSR), which resolves interference by routing internal signals instead of performing parameter-space merge. Specifically, SSR first constructs a unified subspace by concatenating candidate LoRAs along the rank dimension. Next, SSR employs an inverse correlation matrix to decorrelate mixed signals within this space. Finally, a directional guide matrix steers these purified signals into their respective task-specific subspaces. We provide a rigorous theoretical analysis proving that SSR aligns with the Ordinary Least Squares (OLS) solution, thereby ensuring mathematical optimality. We utilize the additivity of sufficient statistics to design a streaming algorithm. This enables on-the-fly updates that significantly reduce memory overhead and computation time. Extensive experiments validate that SSR significantly outperforms state-of-the-art methods while maintaining comparable efficiency. Code is available at https://github.com/nagara214/SSR-Merge.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 50 of its 56 references that are in the map. First posted on arXiv 9 Jun 2026.
Builds on
50 of 56 references.
Closest ancestors · 3
Same direction, newest first
- 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
- 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
- 2020Denoising Diffusion Probabilistic Models19 Jun
Other references · 47
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
- 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
35 more
- 2022Editing Models with Task Arithmeticcited by 1,120
- 2023TIES-Merging: Resolving Interference When Merging Modelscited by 818
- 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 694
- 2021Merging Models with Fisher-Weighted Averagingcited by 621
- 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 471
- 2022Dataless Knowledge Fusion by Merging Weights of Language Modelscited by 387
- 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 364
- 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Modelscited by 261
- 2023AdaMerging: Adaptive Model Merging for Multi-Task Learningcited by 252
- 2023ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAscited by 194
- 2024Mixture of LoRA Expertscited by 181
- 2023ZipIt! Merging Models from Different Tasks without Trainingcited by 179
- 2021UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuningcited by 162
- 2024Task Singular Vectors: Reducing Task Interference in Model Mergingcited by 147
- 2024Implicit Style-Content Separation using B-LoRAcited by 135
- 2023Model Breadcrumbs: Scaling Multi-Task Model Merging with Sparse Maskscited by 133
- 2024Localizing Task Information for Improved Model Merging and Compressioncited by 131
- 2024Model merging with SVD to tie the Knotscited by 122
- 2024Twin-Merging: Dynamic Integration of Modular Expertise in Model Mergingcited by 116
- 2024EMR-Merging: Tuning-Free High-Performance Model Mergingcited by 110
- 2024Representation Surgery for Multi-Task Model Mergingcited by 96
- 2025No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspacescited by 96
- 2024LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wildcited by 68
- 2024LoRA-Composer: Leveraging Low-Rank Adaptation for Multi-Concept Customization in Training-Free Diffusion Modelscited by 55
- 2025K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAscited by 48
- 2024LoRA-Flow: Dynamic LoRA Fusion for Large Language Models in Generative Taskscited by 39
- 2025Modeling Multi-Task Model Merging as Adaptive Projective Gradient Descentcited by 39
- 2023Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusioncited by 37
- 2025Accurate and Efficient Low-Rank Model Merging in Core Spacecited by 29
- 2025RobustMerge: Parameter-Efficient Model Merging for MLLMs with Direction Robustnesscited by 26
- 2024LoRA.rar: Learning to Merge LoRAs via Hypernetworks for Subject-Style Conditioned Image Generationcited by 19
- 2024IterIS: Iterative Inference-Solving Alignment for LoRA Mergingcited by 11
- 2025DuoLoRA : Cycle-consistent and Rank-disentangled Content-Style Personalizationcited by 10
- 2021OpenFed: A Comprehensive and Versatile Open-Source Federated Learning Frameworkcited by 5
- 2025SE-Merging: A Self-Enhanced Approach for Dynamic Model Mergingcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024Mixture-of-Subspaces in Low-Rank Adaptation
- 2026Discovering Undisclosed Concepts in LoRA Adapters for Diffusion-Based AIGC Models
- 2026Filtering Memorization from Parameter-Space in Diffusion Models
- 2026Exploring Data-Free LoRA Transferability for Video Diffusion Models
- 2025Revisiting LoRA through the Lens of Parameter Redundancy: Spectral Encoding Helps
- 2026Principal-timestep Restricted Init via Sparse Matrix-decomposition in Flow-matching
- 2023LoRA-Enhanced Distillation on Guided Diffusion Models
- 2025CDM-QTA: Quantized Training Acceleration for Efficient LoRA Fine-Tuning of Diffusion Model