# An Exploratory Study on Fine-Tuning Large Language Models for Secure Code Generation

arXiv: https://arxiv.org/abs/2408.09078 · HTML: https://mlsanity.com/paper/2408.09078/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Aug 2024.

## Built on by (most cited first, 9 of 9)

- [RESCUE: Retrieval Augmented Secure Code Generation](https://mlsanity.com/paper/2510.18204/) (2025)
- [Seeker: Towards Exception Safety Code Generation with Intermediate Language Agents Framework](https://mlsanity.com/paper/2412.11713/) (2024)
- [Prada: Black-Box LLM Adaptation with Private Data on Resource-Constrained Devices](https://mlsanity.com/paper/2503.14932/) (2025)
- [Secure-Instruct: An Automated Pipeline for Synthesizing Instruction-Tuning Datasets Using LLMs for Secure Code Generation](https://mlsanity.com/paper/2510.07189/) (2025)
- [BabelCoder: Agentic Code Translation with Specification Alignment](https://mlsanity.com/paper/2512.06902/) (2025)
- [A Study on the Impact of Fault localization Granularity for Repository-Scale Code Repair Tasks](https://mlsanity.com/paper/2604.00167/) (2026)
- [Social Bias in LLM-Generated Code: Benchmark and Mitigation](https://mlsanity.com/paper/2605.00382/) (2026)
- [LIA: Supervised Fine-Tuning of Large Language Models for Automatic Issue Assignment](https://mlsanity.com/paper/2601.01780/) (2026)
- [Security in LLM-Assisted Coding: A Systematic Literature Review](https://mlsanity.com/paper/2610.15482/) (2026)

## Abstract (arXiv)

AI-powered coding assistants such as GitHub's Copilot and OpenAI's ChatGPT have achieved notable success in automating code generation. However, these tools rely on pre-trained Large Language Models (LLMs) that are typically trained on human-written code sourced from open-source project hosting sites like GitHub, which often contains inherent security vulnerabilities. These vulnerabilities may then be mirrored in the code generated by these LLMs, a critical risk revealed and highlighted by recent empirical studies. In this work, we present an exploratory study on whether fine-tuning pre-trained LLMs on datasets of vulnerability-fixing commits can promote secure code generation. We explored full fine-tuning and two parameter-efficient fine-tuning techniques (LoRA and IA3) on four pre-trained LLMs for code generation. We crawled a fine-tuning dataset (14,622 C/C++ files) for secure code generation by collecting code fixes of confirmed vulnerabilities from open-source repositories. Our evaluation dataset comprises 52 vulnerability scenarios designed to cover the top most dangerous C/C++ CWEs. Our exploration reveals that fine-tuning LLMs using PEFT techniques can enhance secure code generation. We observe maximum improvements in security of 6.4% in C language and 5.0% in C++ language. In addition, we compared between the fine-tuning approaches and the prompt-based approaches. The LoRA-tuned models outperform the prompt-based approaches in secure code generation. We found that fine-tuning with function-level and block-level datasets achieves the best secure code generation performance, compared to the alternatives (file-level and line-level).