mlsanityData to 14 Oct 2026

IntLoRA: Integral Low-rank Adaptation of Quantized Diffusion Models

arXiv 2410.21759 ↗First posted 29 Oct 2024cs.CVcited by 10

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Fine-tuning pre-trained diffusion models under limited budgets has gained great success. In particular, the recent advances that directly fine-tune the quantized weights using Low-rank Adaptation (LoRA) further reduces training costs. Despite these progress, we point out that existing adaptation recipes are not inference-efficient. Specifically, additional post-training quantization (PTQ) on tuned weights is needed during deployment, which results in noticeable performance drop when the bit-width is low. Based on this observation, we introduce IntLoRA, which adapts quantized diffusion models with integer-type low-rank parameters, to include inference efficiency during tuning. Specifically, IntLoRA enables pre-trained weights to remain quantized during training, facilitating fine-tuning on consumer-level GPUs. During inference, IntLoRA weights can be seamlessly merged into pre-trained weights to directly obtain quantized downstream weights without PTQ. Extensive experiments show our IntLoRA achieves significant speedup on both training and inference without losing performance.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 48 of its 55 references that are in the map. First posted on arXiv 29 Oct 2024.

Builds on

48 of 55 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models27 Nov
  2. 2023EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models5 Oct
  3. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  4. 2023PTQD: Accurate Post-Training Quantization for Diffusion Models18 May
  5. 2023Q-Diffusion: Quantizing Diffusion Models8 Feb
  6. 2022Post-training Quantization on Diffusion Models28 Nov
  7. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec

Other references · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  7. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  11. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  12. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
29 more
  1. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  2. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  3. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  4. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  5. 2022Visual Prompt Tuningcited by 1,681
  6. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  7. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  8. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 991
  9. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
  10. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  11. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 719
  12. 2019Learned Step Size Quantizationcited by 628
  13. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  14. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  15. 2021A White Paper on Neural Network Quantizationcited by 472
  16. 2022Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuningcited by 294
  17. 2023Controlling Text-to-Image Diffusion by Orthogonal Finetuningcited by 208
  18. 2023StyleDrop: Text-to-Image Generation in Any Stylecited by 199
  19. 2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 180
  20. 2023QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Modelscited by 155
  21. 2022FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformercited by 133
  22. 2023Parameter-Efficient Orthogonal Finetuning via Butterfly Factorizationcited by 114
  23. 2022Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformercited by 108
  24. 2019Loss Aware Post-training Quantizationcited by 97
  25. 2023One-for-All: Generalized LoRA for Parameter-Efficient Fine-tuningcited by 85
  26. 2022PD-Quant: Post-Training Quantization based on Prediction Difference Metriccited by 73
  27. 2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 63
  28. 2024Accurate LoRA-Finetuning Quantization of LLMs via Information Retentioncited by 41
  29. 2023Q-DETR: An Efficient Low-Bit Quantized Detection Transformercited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 4.

10 papers.

  1. 2025EEdit: Rethinking the Spatial and Temporal Redundancy for Efficient Image Editingcited by 45
  2. 2025FastVAR: Linear Visual Autoregressive Modeling via Cached Token Pruningcited by 34
  3. 2025Diffusion Model Quantization: A Reviewcited by 6
  4. 2026AdaCluster: Adaptive Query-Key Clustering for Sparse Attention in Video Generationcited by 6
  5. 2025LoTA-QAF: Lossless Ternary Adaptation for Quantization-Aware Fine-Tuningcited by 5
  6. 2026Q-Drift: Quantization-Aware Drift Correction for Diffusion Model Samplingcited by 4
  7. 2024Filterless Snapshot Hyperspectral Imaging using Guided Patch Diffusioncited by 1
  8. 2025FALQON: Accelerating LoRA Fine-tuning with Low-Bit Floating-Point Arithmeticcited by 1
  9. 2026LoRA-Key: User-Centric LoRA Watermarking for Text-to-Image Diffusion Modelscited by 1
  10. 2026In-Cell Learning: Language Models That Update Their Own Weights in Sequence Without Changing the File They Shipcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Memory-Efficient Fine-Tuning for Quantized Diffusion Model
  2. 2025CDM-QTA: Quantized Training Acceleration for Efficient LoRA Fine-Tuning of Diffusion Model
  3. 2024DilateQuant: Accurate and Efficient Diffusion Quantization via Weight Dilation
  4. 2025Pioneering 4-Bit FP Quantization for Diffusion Models: Mixup-Sign Quantization and Timestep-Aware Fine-Tuning
  5. 2024Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers
  6. 2025Zero-Shot Adaptation of Parameter-Efficient Fine-Tuning in Diffusion Models
  7. 2024QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuning
  8. 2024TCAQ-DM: Timestep-Channel Adaptive Quantization for Diffusion Models