# Neural Transfer Learning for Repairing Security Vulnerabilities in C Code

arXiv: https://arxiv.org/abs/2104.08308 · HTML: https://mlsanity.com/paper/2104.08308/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 86 references that are in the map.
- First posted on arXiv 16 Apr 2021.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [A Survey on Deep Transfer Learning](https://mlsanity.com/paper/1808.01974/) (2018)
- [Rethinking ImageNet Pre-training](https://mlsanity.com/paper/1811.08883/) (2018)
- [Unified Pre-training for Program Understanding and Generation](https://mlsanity.com/paper/2103.06333/) (2021)
- [OpenNMT: Open-source Toolkit for Neural Machine Translation](https://mlsanity.com/paper/1709.03815/) (2017)
- [A Survey of Machine Learning for Big Code and Naturalness](https://mlsanity.com/paper/1709.06182/) (2017)
- [Incorporating Copying Mechanism in Sequence-to-Sequence Learning](https://mlsanity.com/paper/1603.06393/) (2016)
- [Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning](https://mlsanity.com/paper/1602.03409/) (2016)
- [code2seq: Generating Sequences from Structured Representations of Code](https://mlsanity.com/paper/1808.01400/) (2018)
- [SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repair](https://mlsanity.com/paper/1901.01808/) (2018)
- [The Adverse Effects of Code Duplication in Machine Learning Models of Code](https://mlsanity.com/paper/1812.06469/) (2018)
- [Automated Vulnerability Detection in Source Code Using Deep Representation Learning](https://mlsanity.com/paper/1807.04320/) (2018)
- [An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation](https://mlsanity.com/paper/1812.08693/) (2018)
- [Graph-based, Self-Supervised Program Repair from Diagnostic Feedback](https://mlsanity.com/paper/2005.10636/) (2020)
- [CODIT: Code Editing with Tree-Based Neural Models](https://mlsanity.com/paper/1810.00314/) (2018)
- [Learning to Represent Edits](https://mlsanity.com/paper/1810.13337/) (2018)
- [A Manually-Curated Dataset of Fixes to Vulnerabilities of Open-Source Software](https://mlsanity.com/paper/1902.02595/) (2019)
- [Learning to Fix Build Errors with Graph2Diff Neural Networks](https://mlsanity.com/paper/1911.01205/) (2019)

## Built on by (most cited first, 30 of 36)

- [Large Language Models for Code: Security Hardening and Adversarial Testing](https://mlsanity.com/paper/2302.05319/) (2023)
- [How Effective Are Neural Networks for Fixing Security Vulnerabilities](https://mlsanity.com/paper/2305.18607/) (2023)
- [RepairLLaMA: Efficient Representations and Fine-Tuned Adapters for Program Repair](https://mlsanity.com/paper/2312.15698/) (2023)
- [A New Era in Software Security: Towards Self-Healing Software via Large Language Models and Formal Verification](https://mlsanity.com/paper/2305.14752/) (2023)
- [HDLdebugger: Streamlining HDL debugging with Large Language Models](https://mlsanity.com/paper/2403.11671/) (2024)
- [LLM-Powered Code Vulnerability Repair with Reinforcement Learning and Semantic Reward](https://mlsanity.com/paper/2401.03374/) (2024)
- [On Distribution Shift in Learning-based Bug Detectors](https://mlsanity.com/paper/2204.10049/) (2022)
- [Enhancing Large Language Models for Secure Code Generation: A Dataset-driven Study on Vulnerability Mitigation](https://mlsanity.com/paper/2310.16263/) (2023)
- [RAPGen: An Approach for Fixing Code Inefficiencies in Zero-Shot](https://mlsanity.com/paper/2306.17077/) (2023)
- [KNOD: Domain Knowledge Distilled Tree Decoder for Automated Program Repair](https://mlsanity.com/paper/2302.01857/) (2023)
- [SecureFalcon: Are We There Yet in Automated Software Vulnerability Detection with LLMs?](https://mlsanity.com/paper/2307.06616/) (2023)
- [Enhanced Automated Code Vulnerability Repair using Large Language Models](https://mlsanity.com/paper/2401.03741/) (2024)
- [Code Security Vulnerability Repair Using Reinforcement Learning with Large Language Models](https://mlsanity.com/paper/2401.07031/) (2024)
- [From Vulnerabilities to Remediation: A Systematic Literature Review of LLMs in Code Security](https://mlsanity.com/paper/2412.15004/) (2024)
- [Large Language Models for Security Operations Centers: A Comprehensive Survey](https://mlsanity.com/paper/2509.10858/) (2025)
- [Automated Program Repair: Emerging trends pose and expose problems for benchmarks](https://mlsanity.com/paper/2405.05455/) (2024)
- [Can GPT-O1 Kill All Bugs? An Evaluation of GPT-Family LLMs on QuixBugs](https://mlsanity.com/paper/2409.10033/) (2024)
- [Agent That Debugs: Dynamic State-Guided Vulnerability Repair](https://mlsanity.com/paper/2504.07634/) (2025)
- [A Survey of Source Code Representations for Machine Learning-Based Cybersecurity Tasks](https://mlsanity.com/paper/2403.10646/) (2024)
- [AI for DevSecOps: A Landscape and Future Opportunities](https://mlsanity.com/paper/2404.04839/) (2024)
- [Enhancing Source Code Security with LLMs: Demystifying The Challenges and Generating Reliable Repairs](https://mlsanity.com/paper/2409.00571/) (2024)
- [Self-Supervised Learning to Prove Equivalence Between Straight-Line Programs via Rewrite Rules](https://mlsanity.com/paper/2109.10476/) (2021)
- [SCoPE: Evaluating LLMs for Software Vulnerability Detection](https://mlsanity.com/paper/2407.14372/) (2024)
- [Improving Retrieval-Augmented Deep Assertion Generation via Joint Training](https://mlsanity.com/paper/2502.10696/) (2025)
- [VulnResolver: A Hybrid Agent Framework for LLM-Based Automated Vulnerability Issue Resolution](https://mlsanity.com/paper/2601.13933/) (2026)
- [EvoRepair: Enhancing Vulnerability Repair Agents Through Experience-Based Self-Evolution](https://mlsanity.com/paper/2605.30105/) (2026)
- [Redundancy and Concept Analysis for Code-trained Language Models](https://mlsanity.com/paper/2305.00875/) (2023)
- [An Exploratory Literature Study on Sharing and Energy Use of Language Models for Source Code](https://mlsanity.com/paper/2307.02443/) (2023)
- [Towards Automatically Addressing Self-Admitted Technical Debt: How Far Are We?](https://mlsanity.com/paper/2308.08943/) (2023)
- [Empirical Study of Code Large Language Models for Binary Security Patch Detection](https://mlsanity.com/paper/2509.06052/) (2025)

## Abstract (arXiv)

In this paper, we address the problem of automatic repair of software vulnerabilities with deep learning. The major problem with data-driven vulnerability repair is that the few existing datasets of known confirmed vulnerabilities consist of only a few thousand examples. However, training a deep learning model often requires hundreds of thousands of examples. In this work, we leverage the intuition that the bug fixing task and the vulnerability fixing task are related and that the knowledge learned from bug fixes can be transferred to fixing vulnerabilities. In the machine learning community, this technique is called transfer learning. In this paper, we propose an approach for repairing security vulnerabilities named VRepair which is based on transfer learning. VRepair is first trained on a large bug fix corpus and is then tuned on a vulnerability fix dataset, which is an order of magnitude smaller. In our experiments, we show that a model trained only on a bug fix corpus can already fix some vulnerabilities. Then, we demonstrate that transfer learning improves the ability to repair vulnerable C functions. We also show that the transfer learning model performs better than a model trained with a denoising task and fine-tuned on the vulnerability fixing task. To sum up, this paper shows that transfer learning works well for repairing security vulnerabilities in C compared to learning on a small dataset.