mlsanityData to 14 Oct 2026

D2LoRA: Data-Driven LoRA Initialization for Low Resource Tasks

arXiv 2503.18089 ↗First posted 23 Mar 2025cs.CLcited by 0

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Tuning large language models is essential for optimizing their performance across diverse applications, particularly in scenarios with limited data availability. Tuning large language models in scarce data scenarios is crucial, particularly given that the convergence speed of the LoRA method is lower than that of full fine-tuning. In this paper, we present an analysis of post-training methods including Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Odds Ratio Preference Optimization (ORPO) within the context of task-specific learning using the LoRA method. Next we introduce D2LoRA, a data-driven approach for initializing LoRA metrics that enhances training efficiency, especially in limited-data settings. Our experiments compare D2LoRA with vanilla LoRA in terms of performance and catastrophic forgetting under extremely data-constrained conditions. The results demonstrate that D2LoRA achieves a 1% improvement GSM8K benchmark and a 2-point improvement in ROUGE score in title generation tasks. D2LoRA facilitates the adaptation of LLMs to multiple tasks even when task-specific data is scarce, thereby reducing training expenses and offering data cost.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 11 of its 17 references that are in the map. First posted on arXiv 23 Mar 2025.

Builds on

11 of 17 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024ORPO: Monolithic Preference Optimization without Reference Model12 Mar
  2. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 9

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  6. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  7. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  8. 2024Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMscited by 273
  9. 2024The Impact of Initialization on LoRA Finetuning Dynamicscited by 54

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters
  2. 2025LoRA Is Slower Than You Think
  3. 2026Learning Rate Matters: Vanilla LoRA May Suffice for LLM Fine-tuning
  4. 2026TLoRA: Task-aware Low Rank Adaptation of Large Language Models
  5. 2024LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Report
  6. 2024Mixture-of-LoRAs: An Efficient Multitask Tuning for Large Language Models
  7. 2025Dual LoRA: Enhancing LoRA with Magnitude and Direction Updates
  8. 2025Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment