# Exploring Data-Free LoRA Transferability for Video Diffusion Models

arXiv: https://arxiv.org/abs/2605.01929 · HTML: https://mlsanity.com/paper/2605.01929/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 26 of its 37 references that are in the map.
- First posted on arXiv 3 May 2026.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 26 of 26)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [WorldSimBench: Towards Video Generation Models as World Simulators](https://mlsanity.com/paper/2410.18072/) (2024)
- [Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion](https://mlsanity.com/paper/2506.08009/) (2025)
- [VACE: All-in-One Video Creation and Editing](https://mlsanity.com/paper/2503.07598/) (2025)
- [From Slow Bidirectional to Fast Autoregressive Video Diffusion Models](https://mlsanity.com/paper/2412.07772/) (2024)
- [Improving Video Generation with Human Feedback](https://mlsanity.com/paper/2501.13918/) (2025)
- [Rolling Forcing: Autoregressive Long Video Diffusion in Real Time](https://mlsanity.com/paper/2509.25161/) (2025)
- [Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity](https://mlsanity.com/paper/2502.01776/) (2025)
- [Diffusion Adversarial Post-Training for One-Step Video Generation](https://mlsanity.com/paper/2501.08316/) (2025)
- [Long-Context Autoregressive Video Modeling with Next-Frame Prediction](https://mlsanity.com/paper/2503.19325/) (2025)
- [LoRA vs Full Fine-tuning: An Illusion of Equivalence](https://mlsanity.com/paper/2410.21228/) (2024)
- [VSA: Faster Video Diffusion with Trainable Sparse Attention](https://mlsanity.com/paper/2505.13389/) (2025)
- [Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillation](https://mlsanity.com/paper/2512.04678/) (2025)
- [Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharing](https://mlsanity.com/paper/2411.16375/) (2024)
- [DOLLAR: Few-Step Video Generation via Distillation and Latent Reward Optimization](https://mlsanity.com/paper/2412.15689/) (2024)
- [Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment](https://mlsanity.com/paper/2502.16894/) (2025)
- [X-Adapter: Adding Universal Compatibility of Plugins for Upgraded Diffusion Model](https://mlsanity.com/paper/2312.02238/) (2023)
- [See Further for Parameter Efficient Fine-tuning by Standing on the Shoulders of Decomposition](https://mlsanity.com/paper/2407.05417/) (2024)
- [textitTrans-LoRA: towards data-free Transferable Parameter Efficient Finetuning](https://mlsanity.com/paper/2405.17258/) (2024)
- [Efficient Video Diffusion Models: Advancements and Challenges](https://mlsanity.com/paper/2604.15911/) (2026)
- [Weight Spectra Induced Efficient Model Adaptation](https://mlsanity.com/paper/2505.23099/) (2025)
- [Zero-Shot Adaptation of Parameter-Efficient Fine-Tuning in Diffusion Models](https://mlsanity.com/paper/2506.04244/) (2025)

## Built on by (most cited first, 3 of 3)

- [DART: Distillation-Aware Reparameterization for Training-Free LoRA Reuse in Few-Step Video Diffusion Models](https://mlsanity.com/paper/2609.20051/) (2026)
- [G3-LoRA: Organizing Reward-Weighted Video Data with Gradient-Guided Grouped LoRA](https://mlsanity.com/paper/2609.35189/) (2026)
- [LongLive-Plug: Once-for-All Distillation for Video Generation](https://mlsanity.com/paper/2609.38154/) (2026)

## Abstract (arXiv)

Video diffusion models leveraging step distillation or causal distillation have achieved remarkable performance. However, adapting existing LoRAs to these variants remains a critical challenge due to weight space mismatches. We observe that direct application leads to style degradation and structural collapse, yet the underlying mechanisms remain poorly understood. To fill this gap, we delve into the weight space and identify that the incompatibility stems from spectral interference within shared functional clusters defined over singular subspaces. Specifically, our analysis reveals that while both paradigms respect spectral rigidity, they establish conflicting routing pathways that clash through constructive overload or destructive cancellation. To address this issue, we propose Cluster-Aware Spectral Arbitration (CASA), a data-free framework that dynamically arbitrates between safeguarding the target's manifold and restoring LoRA alignment based on spectral density. Extensive experiments demonstrate that CASA effectively mitigates artifacts and revives LoRA functionality. Our code is available at https://github.com/Noahwangyuchen/CASA