# An empirical study of LoRA-based fine-tuning of large language models for automated test case generation

arXiv: https://arxiv.org/abs/2604.06946 · HTML: https://mlsanity.com/paper/2604.06946/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 51 references that are in the map.
- First posted on arXiv 8 Apr 2026.

## Builds on (most cited first, 24 of 24)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints](https://mlsanity.com/paper/2305.13245/) (2023)
- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [A Survey of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2312.14925/) (2023)
- [Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models](https://mlsanity.com/paper/2205.10770/) (2022)
- [An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation](https://mlsanity.com/paper/2302.06527/) (2023)
- [Large Language Models for Software Engineering: Survey and Open Problems](https://mlsanity.com/paper/2310.03533/) (2023)
- [Software Testing with Large Language Models: Survey, Landscape, and Vision](https://mlsanity.com/paper/2307.07221/) (2023)
- [Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluation](https://mlsanity.com/paper/2407.10817/) (2024)
- [Towards an Understanding of Large Language Models in Software Engineering Tasks](https://mlsanity.com/paper/2308.11396/) (2023)
- [Effective Test Generation Using Pre-trained Large Language Models and Mutation Testing](https://mlsanity.com/paper/2308.16557/) (2023)
- [Understanding the Performance and Estimating the Cost of LLM Fine-Tuning](https://mlsanity.com/paper/2408.04693/) (2024)
- [TESTEVAL: Benchmarking Large Language Models for Test Case Generation](https://mlsanity.com/paper/2406.04531/) (2024)
- [Low-Rank Adaptation for Foundation Models: A Comprehensive Review](https://mlsanity.com/paper/2501.00365/) (2024)
- [Unveiling the Generalization Power of Fine-Tuned Large Language Models](https://mlsanity.com/paper/2403.09162/) (2024)
- [A3Test: Assertion-Augmented Automated Test Case Generation](https://mlsanity.com/paper/2302.10352/) (2023)
- [Assessing and Advancing Benchmarks for Evaluating Large Language Models in Software Engineering Tasks](https://mlsanity.com/paper/2505.08903/) (2025)
- [Grade Like a Human: Rethinking Automated Assessment with Large Language Models](https://mlsanity.com/paper/2405.19694/) (2024)
- [Large Language Models (LLMs) for Requirements Engineering (RE): A Systematic Literature Review](https://mlsanity.com/paper/2509.11446/) (2025)
- [Enhancing Large Language Models for Text-to-Testcase Generation](https://mlsanity.com/paper/2402.11910/) (2024)
- [Requirements-Based Test Generation: A Comprehensive Survey](https://mlsanity.com/paper/2505.02015/) (2025)
- [A critical review of methods and challenges in large language models](https://mlsanity.com/paper/2404.11973/) (2024)

## Abstract (arXiv)

Automated test case generation from natural language requirements remains a challenging problem in software engineering due to the ambiguity of requirements and the need to produce structured, executable test artifacts. Recent advances in LLMs have shown promise in addressing this task; however, their effectiveness depends on task-specific adaptation and efficient fine-tuning strategies. In this paper, we present a comprehensive empirical study on the use of parameter-efficient fine-tuning, specifically LoRA, for requirement-based test case generation. We evaluate multiple LLM families, including open-source and proprietary models, under a unified experimental pipeline. The study systematically explores the impact of key LoRA hyperparameters, including rank, scaling factor, and dropout, on downstream performance. We propose an automated evaluation framework based on GPT-4o, which assesses generated test cases across nine quality dimensions. Experimental results demonstrate that LoRA-based fine-tuning significantly improves the performance of all open-source models, with Ministral-8B achieving the best results among them. Furthermore, we show that a fine-tuned 8B open-source model can achieve performance comparable to pre-fine-tuned GPT-4.1 models, highlighting the effectiveness of parameter-efficient adaptation. While GPT-4.1 models achieve the highest overall performance, the performance gap between proprietary and open-source models is substantially reduced after fine-tuning. These findings provide important insights into model selection, fine-tuning strategies, and evaluation methods for automated test generation. In particular, they demonstrate that cost-efficient, locally deployable open-source models can serve as viable alternatives to proprietary systems when combined with well-designed fine-tuning approaches.