mlsanityData to 14 Oct 2026

Enhanced Automated Code Vulnerability Repair using Large Language Models

arXiv 2401.03741 ↗First posted 8 Jan 2024cs.SE cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

This research addresses the complex challenge of automated repair of code vulnerabilities, vital for enhancing digital security in an increasingly technology-driven world. The study introduces a novel and efficient format for the representation of code modification, using advanced Large Language Models (LLMs) such as Code Llama and Mistral. These models, fine-tuned on datasets featuring C code vulnerabilities, significantly improve the accuracy and adaptability of automated code repair techniques. A key finding is the enhanced repair accuracy of these models when compared to previous methods such as VulRepair, which underscores their practical utility and efficiency. The research also offers a critical assessment of current evaluation metrics, such as perfect predictions, and their limitations in reflecting the true capabilities of automated repair models in real-world scenarios. Following this, it underscores the importance of using test datasets devoid of train samples, emphasizing the need for dataset integrity to enhance the effectiveness of LLMs in code repair tasks. The significance of this work is its contribution to digital security, setting new standards for automated code vulnerability repair and paving the way for future advancements in the fields of cybersecurity and artificial intelligence. The study does not only highlight the potential of LLMs in enhancing code security but also fosters further exploration and research in these crucial areas.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 46 of its 76 references that are in the map. First posted on arXiv 8 Jan 2024.

Builds on

46 of 76 references.

References · 46

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  7. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  9. 2020Scaling Laws for Neural Language Modelscited by 6,586
  10. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  11. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  12. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
34 more
  1. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  2. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  3. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  4. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  5. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  6. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  7. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
  8. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  9. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
  10. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 744
  11. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 609
  12. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 559
  13. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 498
  14. 2021Unified Pre-training for Program Understanding and Generationcited by 366
  15. 2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 361
  16. 2022UL2: Unifying Language Learning Paradigmscited by 295
  17. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 251
  18. 2022Practical Program Repair in the Era of Large Pre-trained Language Modelscited by 133
  19. 2023CodeGen2: Lessons for Training LLMs on Programming and Natural Languagescited by 125
  20. 2023NEFTune: Noisy Embeddings Improve Instruction Finetuningcited by 100
  21. 2021CVEfixes: Automated Collection of Vulnerabilities and Their Fixes from Open-Source Softwarecited by 91
  22. 2019TBar: Revisiting Template-based Automated Program Repaircited by 65
  23. 2023A Survey of Learning-based Automated Program Repaircited by 55
  24. 2023How Effective Are Neural Networks for Fixing Security Vulnerabilitiescited by 52
  25. 2021Neural Transfer Learning for Repairing Security Vulnerabilities in C Codecited by 36
  26. 2018Nopol: Automatic Repair of Conditional Statement Bugs in Java Programscited by 32
  27. 2017ARJA: Automated Repair of Java Programs via Multi-Objective Genetic Programmingcited by 26
  28. 2018FixMiner: Mining Relevant Fix Patterns for Automated Program Repaircited by 22
  29. 2024LLM-Powered Code Vulnerability Repair with Reinforcement Learning and Semantic Rewardcited by 16
  30. 2018Learning to Repair Software Vulnerabilities with Generative Adversarial Networkscited by 12
  31. 2022Improving Automated Program Repair with Domain Adaptationcited by 10
  32. 2017Ultra-Large Repair Search Space with Automatically Mined Templates: the Cardumen Mode of Astorcited by 9
  33. 2018The Coming Era of AlphaHacking? A Survey of Automatic Software Vulnerability Detection, Exploitation and Patching Techniquescited by 5
  34. 2022Program Repair: Automated vs. Manualcited by 2

Built on by

Cited by year

Citing papers per year: 2024: 4; 2025: 6.

10 papers.