mlsanityData to 7 Oct 2026

Parameter Efficient Fine Tuning: A Comprehensive Analysis Across Applications

arXiv 2404.13506 ↗First posted 21 Apr 2024cs.LG cs.AI cs.CLcited by 25

Not on the map. Why?

Abstract (arXiv)

The rise of deep learning has marked significant progress in fields such as computer vision, natural language processing, and medical imaging, primarily through the adaptation of pre-trained models for specific tasks. Traditional fine-tuning methods, involving adjustments to all parameters, face challenges due to high computational and memory demands. This has led to the development of Parameter Efficient Fine-Tuning (PEFT) techniques, which selectively update parameters to balance computational efficiency with performance. This review examines PEFT approaches, offering a detailed comparison of various strategies highlighting applications across different domains, including text generation, medical imaging, protein modeling, and speech synthesis. By assessing the effectiveness of PEFT methods in reducing computational load, speeding up training, and lowering memory usage, this paper contributes to making deep learning more accessible and adaptable, facilitating its wider application and encouraging innovation in model optimization. Ultimately, the paper aims to contribute towards insights into PEFT's evolving landscape, guiding researchers and practitioners in overcoming the limitations of conventional fine-tuning approaches.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 35 of its 52 references that are in the map. First posted on arXiv 21 Apr 2024.

Builds on

35 of 52 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  5. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  6. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  7. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  8. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  9. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  10. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  11. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
  12. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
23 more
  1. 2023A Comprehensive Overview of Large Language Modelscited by 639
  2. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 626
  3. 2019Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Datacited by 532
  4. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 386
  5. 2023LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Modelscited by 317
  6. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 263
  7. 2020Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgettingcited by 169
  8. 2023Full Parameter Fine-tuning for Large Language Models with Limited Resourcescited by 134
  9. 2023End-to-End Speech Recognition: A Surveycited by 127
  10. 2022Parameter-efficient Model Adaptation for Vision Transformerscited by 81
  11. 2023Parameter-Efficient Fine-Tuning for Medical Image Analysis: The Missed Opportunitycited by 54
  12. 2021Exploring Wav2vec 2.0 fine-tuning for improved speech emotion recognitioncited by 47
  13. 2023LLaMA-Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efficient Fine-Tuningcited by 47
  14. 2023Point-PEFT: Parameter-Efficient Fine-Tuning for 3D Pre-trained Modelscited by 42
  15. 2021Controlling Conditional Language Models without Catastrophic Forgettingcited by 32
  16. 2023FairTune: Optimizing Parameter Efficient Fine Tuning for Fairness in Medical Image Analysiscited by 23
  17. 2024Faster and Lighter LLMs: A Survey on Current Challenges and Way Forwardcited by 20
  18. 2023Sparsely Shared LoRA on Whisper for Child Speech Recognitioncited by 18
  19. 2023PEFT-SER: On the Use of Parameter Efficient Transfer Learning Approaches For Speech Emotion Recognition Using Pre-trained Speech Modelscited by 17
  20. 2022Simplex Neural Population Learning: Any-Mixture Bayes-Optimality in Symmetric Zero-sum Gamescited by 8
  21. 2021An Empirical Study and Analysis on Open-Set Semi-Supervised Learningcited by 6
  22. 2023TrustSER: On the Trustworthiness of Fine-tuning Pre-trained Speech Embeddings For Speech Emotion Recognitioncited by 3
  23. 2021Topological Simplifications of Hypergraphscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 14; 2026: 5.

25 papers.

Sort:

By area:

  1. 2025FlowerTune: A Cross-Domain Benchmark for Federated Fine-Tuning of Large Language Modelscited by 11
  2. 2024Beyond LoRA: Exploring Efficient Fine-Tuning Techniques for Time Series Foundational Modelscited by 10
  3. 2025IndicMMLU-Pro: Benchmarking Indic Large Language Models on Multi-Task Language Understandingcited by 9
  4. 2025Re-Imagining Multimodal Instruction Tuning: A Representation Viewcited by 9
  5. 2024Decoding the Diversity: A Review of the Indic AI Research Landscapecited by 4
  6. 2025Evolution of meta's llama models and parameter-efficient fine-tuning of large language models: a surveycited by 4
  7. 2024Parameter-Efficient Fine-Tuning of Large Language Models for Unit Test Generation: An Empirical Studycited by 3
  8. 2025KnowledgeSmith: Uncovering Knowledge Updating in LLMs with Model Editing and Unlearningcited by 3
  9. 2024LangSAMP: Language-Script Aware Multilingual Pretrainingcited by 2
  10. 2024Efficient and Private: Memorisation under differentially private parameter-efficient fine-tuning in language modelscited by 2
  11. 2024Unsupervised Human Preference Learningcited by 1
  12. 2025Parameter Efficient Fine-Tuning of Segment Anything Model for Biomedical Imagingcited by 1
13 more
  1. 2025Adaptive Budget Allocation for Orthogonal-Subspace Adapter Tuning in LLMs Continual Learningcited by 1
  2. 20254,500 Seconds: Small Data Training Approaches for Deep UAV Audio Classificationcited by 1
  3. 2025zFLoRA: Zero-Latency Fused Low-Rank Adapterscited by 1
  4. 2025MedPEFT-CL: Dual-Phase Parameter-Efficient Continual Learning with Medical Semantic Adapter and Bidirectional Memory Consolidationcited by 1
  5. 2026Shapelets-Enriched Selective Forecasting using Time Series Foundation Modelscited by 1
  6. 2025Prompt Mechanisms in Medical Imaging: A Comprehensive Surveycited by 0
  7. 2025FLoRA: Fused forward-backward adapters for parameter efficient fine-tuning and reducing inference-time latencies of LLMscited by 0
  8. 2025Loquetier: A Virtualized Multi-LoRA Framework for Unified LLM Fine-tuning and Servingcited by 0
  9. 2025Textual Data Bias Detection and Mitigation -- An Extensible Pipeline with Experimental Evaluationcited by 0
  10. 2026NanoNet: Parameter-Efficient Learning with Label-Scarce Supervision for Lightweight Text Mining Modelcited by 0
  11. 2026Direct Preference Optimization for Chatbot Fine-Tuning: An Empirical Studycited by 0
  12. 2026Energy- and Memory-Efficient PEFT Methods for Personalized On-Device SLMs on Consumer GPUscited by 0
  13. 2026From Manuals to Maintenance: Fine-Tuning MedGemma for Multi-Modal Imaging System Support in Low-Resource Settingscited by 0