mlsanityData to 14 Oct 2026

Optimizing Large Language Models with an Enhanced LoRA Fine-Tuning Algorithm for Efficiency and Robustness in NLP Tasks

arXiv 2412.18729 ↗First posted 25 Dec 2024cs.CL cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

This study proposes a large language model optimization method based on the improved LoRA fine-tuning algorithm, aiming to improve the accuracy and computational efficiency of the model in natural language processing tasks. We fine-tune the large language model through a low-rank adaptation strategy, which significantly reduces the consumption of computing resources while maintaining the powerful capabilities of the pre-trained model. The experiment uses the QQP task as the evaluation scenario. The results show that the improved LoRA algorithm shows significant improvements in accuracy, F1 score, and MCC compared with traditional models such as BERT, Roberta, T5, and GPT-4. In particular, in terms of F1 score and MCC, our model shows stronger robustness and discrimination ability, which proves the potential of the improved LoRA algorithm in fine-tuning large-scale pre-trained models. In addition, this paper also discusses the application prospects of the improved LoRA algorithm in other natural language processing tasks, emphasizing its advantages in multi-task learning and scenarios with limited computing resources. Future research can further optimize the LoRA fine-tuning strategy and expand its application in larger-scale pre-trained models to improve the generalization ability and task adaptability of the model.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 14 of its 22 references that are in the map. First posted on arXiv 25 Dec 2024.

Builds on

14 of 22 references.

References · 14

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessmentcited by 230
  3. 2023LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Modelscited by 196
  4. 2024MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Expertscited by 147
  5. 2021BERT: A Review of Applications in Natural Language Processing and Understandingcited by 109
  6. 2024Chain of LoRA: Efficient Fine-tuning of Language Models via Residual Learningcited by 82
  7. 2023LoRA ensembles for large language model fine-tuningcited by 67
  8. 2024Transforming Multidimensional Time Series into Interpretable Event Sequences for Advanced Data Miningcited by 47
  9. 2024Self-Supervised Graph Neural Networks for Enhanced Feature Extraction in Heterogeneous Information Networkscited by 33
  10. 2024Research on Adverse Drug Reaction Prediction Model Combining Knowledge Graph Embedding and Deep Learningcited by 17
  11. 2024Harnessing LLMs for API Interactions: A Framework for Classification and Synthetic Data Generationcited by 17
  12. 2024LoRA-LiteE: A Computationally Efficient Framework for Chatbot Preference-Tuningcited by 10
2 more
  1. 2024Deep Learning for Cross-Border Transaction Anomaly Detection in Anti-Money Laundering Systemscited by 9
  2. 2024Integrative Analysis of Financial Market Sentiment Using CNN and GRU for Risk Prediction and Alert Systemscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Object Detection for Medical Image Analysis: Insights from the RT-DETR Modelcited by 10
  2. 2025Optimized Unet with Attention Mechanism for Multi-Scale Semantic Segmentationcited by 9
  3. 2025Capsule Network-Based Semantic Intent Modeling for Human-Computer Interactioncited by 5
  4. 2026Open-ended Multi-agent Autocurricula via Visual Inspection of Policies with Multi-modal LLMscited by 0