mlsanityData to 14 Oct 2026

Improving Automated Secure Code Reviews: A Synthetic Dataset for Code Vulnerability Flaws

arXiv 2504.16310 ↗First posted 22 Apr 2025cs.SEcited by 0

On the map: Synthetic data and self-improvement for LLMs → Code and instruction data synthesis

Abstract (arXiv)

Automation of code reviews using AI models has garnered substantial attention in the software engineering community as a strategy to reduce the cost and effort associated with traditional peer review processes. These models are typically trained on extensive datasets of real-world code reviews that address diverse software development concerns, including testing, refactoring, bug fixes, performance optimization, and maintainability improvements. However, a notable limitation of these datasets is the under representation of code vulnerabilities, critical flaws that pose significant security risks, with security-focused reviews comprising a small fraction of the data. This scarcity of vulnerability-specific data restricts the effectiveness of AI models in identifying and commenting on security-critical code. To address this issue, we propose the creation of a synthetic dataset consisting of vulnerability-focused reviews that specifically comment on security flaws. Our approach leverages Large Language Models (LLMs) to generate human-like code review comments for vulnerabilities, using insights derived from code differences and commit messages. To evaluate the usefulness of the generated synthetic dataset, we plan to use it to fine-tune three existing code review models. We anticipate that the synthetic dataset will improve the performance of the original code review models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Code and instruction data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 19 of its 26 references that are in the map. First posted on arXiv 22 Apr 2025.

Builds on

19 of 26 references.

References · 19

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 397
  4. 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 232
  5. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 211
  6. 2022Automating Code Review Activities by Large-Scale Pre-trainingcited by 98
  7. 2022Using Pre-Trained Models to Boost Code Review Automationcited by 63
  8. 2021Towards Automating Code Review Activitiescited by 54
  9. 2023LLaMA-Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efficient Fine-Tuningcited by 47
  10. 2023Exploring the Potential of ChatGPT in Automated Code Refinement: An Empirical Studycited by 44
  11. 2024Synthetic Dialogue Dataset Generation using LLM Agentscited by 39
  12. 2022AUGER: Automatically Generating Review Comments with Pre-training Modelscited by 23
7 more
  1. 2024Code Review Automation: Strengths and Weaknesses of the State of the Artcited by 20
  2. 2024Improving Automated Code Reviews: Learning from Experiencecited by 13
  3. 2022CodeEditor: Learning to Edit Source Code with Pre-trained Modelscited by 12
  4. 2021Why Security Defects Go Unnoticed during Code Reviews? A Case-Control Study of the Chromium OS Projectcited by 5
  5. 2022Mining Code Review Data to Understand Waiting Times Between Acceptance and Merging: An Empirical Analysiscited by 4
  6. 2023Security Defect Detection via Code Review: A Study of the OpenStack and Qt Communitiescited by 4
  7. 2022Software Security during Modern Code Review: The Developer's Perspectivecited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025SecureReviewer: Enhancing Large Language Models for Secure Code Review through Secure-aware Fine-tuning
  2. 2024Software Vulnerability and Functionality Assessment using LLMs
  3. 2025Automated Vulnerability Validation and Verification: A Large Language Model Approach
  4. 2025Bugdar: AI-Augmented Secure Code Review for GitHub Pull Requests
  5. 2025Boosting Vulnerability Detection of LLMs via Curriculum Preference Optimization with Synthetic Reasoning Data
  6. 2026SEVRA-BENCH: Social Engineering of Vulnerabilities in Review Agents
  7. 2026Revisiting Vul-RAG: Reproducibility and Replicability of RAG-based Vulnerability Detection with Open-Weight Models
  8. 2026AgenticSCR: An Autonomous Agentic Secure Code Review for Immature Vulnerabilities Detection