mlsanityData to 14 Oct 2026

AutoLoRA: AutoGuidance Meets Low-Rank Adaptation for Diffusion Models

arXiv 2410.03941 ↗First posted 4 Oct 2024cs.CVcited by 7

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Low-rank adaptation (LoRA) is a fine-tuning technique that can be applied to conditional generative diffusion models. LoRA utilizes a small number of context examples to adapt the model to a specific domain, character, style, or concept. However, due to the limited data utilized during training, the fine-tuned model performance is often characterized by strong context bias and a low degree of variability in the generated images. To solve this issue, we introduce AutoLoRA, a novel guidance technique for diffusion models fine-tuned with the LoRA approach. Inspired by other guidance techniques, AutoLoRA searches for a trade-off between consistency in the domain represented by LoRA weights and sample diversity from the base conditional diffusion model. Moreover, we show that incorporating classifier-free guidance for both LoRA fine-tuned and base models leads to generating samples with higher diversity and better quality. The experimental results for several fine-tuned LoRA domains show superiority over existing guidance techniques on selected metrics.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 39 of its 45 references that are in the map. First posted on arXiv 4 Oct 2024.

Builds on

39 of 45 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRA12 Apr
  2. 2023Text-to-image Diffusion Models in Generative AI: A Survey14 Mar

Other references · 37

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  9. 2022Classifier-Free Diffusion Guidancecited by 5,847
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  11. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  12. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
25 more
  1. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  2. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  3. 2015Variational Inference with Normalizing Flowscited by 2,170
  4. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,416
  5. 2014NICE: Non-linear Independent Components Estimationcited by 1,058
  6. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 594
  7. 2023Universal Guidance for Diffusion Modelscited by 437
  8. 2024LoRA+: Efficient Low Rank Adaptation of Large Modelscited by 377
  9. 2023AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuningcited by 357
  10. 2023VeRA: Vector-based Random Matrix Adaptationcited by 300
  11. 2024Guiding a Diffusion Model with a Bad Version of Itselfcited by 299
  12. 2022DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptationcited by 233
  13. 2021Path Integral Sampler: a stochastic control approach for samplingcited by 181
  14. 2022Improving Sample Quality of Diffusion Models Using Self-Attention Guidancecited by 154
  15. 2022On the Effectiveness of Parameter-Efficient Fine-Tuningcited by 141
  16. 2024Flora: Low-Rank Adapters Are Secretly Gradient Compressorscited by 108
  17. 2022Refining Generative Process with Discriminator Guidance in Score-based Diffusion Modelscited by 100
  18. 2023One-for-All: Generalized LoRA for Parameter-Efficient Fine-tuningcited by 85
  19. 2023Delta-LoRA: Fine-Tuning High-Rank Parameters with the Delta of Low-Rank Matricescited by 74
  20. 2024Riemannian Preconditioned LoRA for Fine-Tuning Foundation Modelscited by 52
  21. 2024LoRA Training in the NTK Regime has No Spurious Local Minimacited by 48
  22. 2024Trajectory Consistency Distillation: Improved Latent Consistency Distillation by Semi-Linear Consistency Function with Trajectory Mappingcited by 46
  23. 2023IncreLoRA: Incremental Parameter Allocation Method for Parameter-Efficient Fine-tuningcited by 45
  24. 2017Learning to Generate Samples from Noise through Infusion Trainingcited by 15
  25. 2024GeoGuide: Geometric guidance of diffusion modelscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 1.

7 papers.

  1. 2024Any-Resolution AI-Generated Image Detection by Spectral Learningcited by 57
  2. 2025Steering Guidance for Personalized Text-to-Image Diffusion Modelscited by 7
  3. 2025UnGuide: Learning to Forget with LoRA-Guided Diffusion Modelscited by 7
  4. 2025AudioMoG: Guiding Audio Generation with Mixture-of-Guidancecited by 5
  5. 2025Memory Self-Regeneration: Uncovering Hidden Knowledge in Unlearned Modelscited by 2
  6. 2025Prompt-aware classifier free guidance for diffusion modelscited by 0
  7. 2026EulerLoRA: Rank-Driven Jump Dynamics for Calibrated Parameter-Efficient Fine-Tuningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024LoRA Diffusion: Zero-Shot LoRA Synthesis for Diffusion Model Personalization
  2. 2026Diffu-LoRA: A Novel Low-Rank Adaptation for Personalized Diffusion Models
  3. 2025T-LoRA: Single Image Diffusion Model Customization Without Overfitting
  4. 2025LoRAtorio: An intrinsic approach to LoRA Skill Composition
  5. 2025LoRAverse: A Submodular Framework to Retrieve Diverse Adapters for Diffusion Models
  6. 2023LoRA-Enhanced Distillation on Guided Diffusion Models
  7. 2025Zero-Shot Adaptation of Parameter-Efficient Fine-Tuning in Diffusion Models
  8. 2025Glance: Accelerating Diffusion Models with 1 Sample