mlsanityData to 14 Oct 2026

SuperLoRA: Parameter-Efficient Unified Adaptation of Multi-Layer Attention Modules

arXiv 2403.11887 ↗First posted 18 Mar 2024cs.CV cs.AI cs.LGcited by 10

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Low-rank adaptation (LoRA) and its variants are widely employed in fine-tuning large models, including large language models for natural language processing and diffusion models for computer vision. This paper proposes a generalized framework called SuperLoRA that unifies and extends different LoRA variants, which can be realized under different hyper-parameter settings. Introducing grouping, folding, shuffling, projecting, and tensor factoring, SuperLoRA offers high flexibility compared with other LoRA variants and demonstrates superior performance for transfer learning tasks especially in the extremely few-parameter regimes.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 30 of its 41 references that are in the map. First posted on arXiv 18 Mar 2024.

Builds on

30 of 41 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluation26 Sep

Other references · 29

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2022Classifier-Free Diffusion Guidancecited by 5,847
  7. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  9. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  10. 2022A ConvNet for the 2020scited by 3,579
  11. 2016Improved Techniques for Training GANscited by 2,970
  12. 2018Demystifying MMD GANscited by 1,293
17 more
  1. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 944
  2. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
  3. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 788
  4. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 719
  5. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 671
  6. 2021Compacter: Efficient Low-Rank Hypercomplex Adapter Layerscited by 460
  7. 2024LoRA+: Efficient Low Rank Adaptation of Large Modelscited by 377
  8. 2020Parameter-Efficient Transfer Learning with Diff Pruningcited by 352
  9. 2022FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformercited by 133
  10. 2022KronA: Parameter Efficient Tuning with Kronecker Adaptercited by 133
  11. 2024Asymmetry in Low-Rank Adapters of Foundation Modelscited by 86
  12. 2022Parameter-efficient Model Adaptation for Vision Transformerscited by 81
  13. 2014Fastfood: Approximate Kernel Expansions in Loglinear Timecited by 74
  14. 2023Revisiting the Parameter Efficiency of Adapters from the Perspective of Precision Redundancycited by 44
  15. 2019The Shape of Data: Intrinsic Distance for Data Distributionscited by 38
  16. 2024LoTR: Low Tensor Rank Weight Adaptationcited by 29
  17. 2023Consolidator: Mergeable Adapter with Grouped Connections for Visual Adaptationcited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 3; 2026: 2.

10 papers.

  1. 2024Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologiescited by 47
  2. 2024Depth-Wise Convolutions in Vision Transformers for Efficient Training on Small Datasetscited by 14
  3. 2025Quantum-PEFT: Ultra parameter-efficient fine-tuningcited by 11
  4. 2024Maintaining Structural Integrity in Parameter Spaces for Parameter Efficient Fine-tuningcited by 5
  5. 2024LoRA-PT: Low-Rank Adapting UNETR for Hippocampus Segmentation Using Principal Tensor Singular Values and Vectorscited by 2
  6. 2024Robust 3D Point Clouds Classification based on Declarative Defenderscited by 2
  7. 2025KARST: Multi-Kernel Kronecker Adaptation with Re-Scaling Transmission for Visual Classificationcited by 1
  8. 2026LORA-CRAFT: Cross-layer Rank Adaptation via Frozen Tucker Decomposition of Pre-trained Attention Weightscited by 1
  9. 2025Regularizing Subspace Redundancy of Low-Rank Adaptationcited by 0
  10. 2026Parameter-Efficient Fine-Tuning of Large Pretrained Models for Instance Segmentation Taskscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Less is More: Resource-Efficient Low-Rank Adaptation
  2. 2025SingLoRA: Low Rank Adaptation Using a Single Matrix
  3. 2026A Unified Study of LoRA Variants: Taxonomy, Review, Codebase, and Empirical Evaluation
  4. 2026TLoRA: Task-aware Low Rank Adaptation of Large Language Models
  5. 2026Not All Directions Matter: Towards Structured and Task-Aware Low-Rank Model Adaptation
  6. 2025RefLoRA: Refactored Low-Rank Adaptation for Efficient Fine-Tuning of Large Models
  7. 2024Flat-LoRA: Low-Rank Adaptation over a Flat Loss Landscape
  8. 2024Low-Rank Few-Shot Adaptation of Vision-Language Models