mlsanityData to 14 Oct 2026

A Note on LoRA

arXiv 2404.05086 ↗First posted 7 Apr 2024cs.LG cs.AI cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

LoRA (Low-Rank Adaptation) has emerged as a preferred method for efficiently adapting Large Language Models (LLMs) with remarkable simplicity and efficacy. This note extends the original LoRA paper by offering new perspectives that were not initially discussed and presents a series of insights for deploying LoRA at scale. Without introducing new experiments, we aim to improve the understanding and application of LoRA.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 10 of its 17 references that are in the map. First posted on arXiv 7 Apr 2024.

Builds on

10 of 17 references.

References · 10

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2016Densely Connected Convolutional Networkscited by 6,720
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  8. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  9. 2023ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAscited by 194
  10. 2022HyperTuning: Toward Adapting Large Language Models without Back-propagationcited by 46

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 6; 2026: 2.

10 papers.

  1. 2024A Survey on LoRA of Large Language Modelscited by 67
  2. 2024VeLoRA: Memory Efficient Training using Rank-1 Sub-Token Projectionscited by 18
  3. 2025PLoP: Precise LoRA Placement for Efficient Finetuning of Large Modelscited by 14
  4. 2025GoRA: Gradient-driven Adaptive Low Rank Adaptationcited by 12
  5. 2026Learning Rate Matters: Vanilla LoRA May Suffice for LLM Fine-tuningcited by 10
  6. 2025Do Bias Benchmarks Generalise? Evidence from Voice-based Evaluation of Gender Bias in SpeechLLMscited by 7
  7. 2025Transformed Low-rank Adaptation via Tensor Decomposition and Its Applications to Text-to-image Modelscited by 3
  8. 2025Task-agnostic Low-rank Residual Adaptation for Efficient Federated Continual Fine-Tuningcited by 1
  9. 2025Mixture of Low Rank Adaptation with Partial Parameter Sharing for Time Series Forecastingcited by 1
  10. 2026A Reference-Free Framework for Evaluating Single-Frame ISP Pipelinescited by 0