mlsanityData to 7 Oct 2026

From Evaluation to Optimisation: Hierarchy-Aware Training Signals for CWE Prediction in Python

arXiv 2607.21069 ↗First posted 23 Jul 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The original ALPHA benchmark introduced a taxonomy-aware penalty for evaluating CWE-level vulnerability prediction in Python and proposed that the penalty could theoretically also serve as a training signal. This paper provides that validation. We compare three delivery mechanisms: supervised fine-tuning, a dual-head classification loss, and reinforcement learning with a dense reward derived from the normalised penalty. We find that supervised approaches consistently regress below the zero-shot baseline under distribution shift, while GRPO succeeds. Our best policy reduces the cumulative ALPHA penalty of Qwen2.5-Coder-7B on Security Hardening and Adversarial Testing (SVEN) dataset by 27.9% under greedy decoding, and by 25.5% under sampled decoding(p = 0.005, Welch's t-test), reaching statistical parity with its 4.5x larger zero-shot teacher. We conclude that the value of a hierarchical penalty as a training signal depends largely on the directness of its delivery.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 42 references that are in the map. First posted on arXiv 23 Jul 2026.

Builds on

32 of 42 references.

References · 32

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  6. 2017Deep reinforcement learning from human preferencescited by 3,846
  7. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  8. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 417
  9. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
  10. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  11. 2019A Study of BFLOAT16 for Deep Learning Trainingcited by 227
  12. 2024MapCoder: Multi-Agent Code Generation for Competitive Problem Solvingcited by 177
20 more
  1. 2023Large Language Models for Code: Security Hardening and Adversarial Testingcited by 149
  2. 2019Making Better Mistakes: Leveraging Class Hierarchies with Deep Networkscited by 112
  3. 2023RLTF: Reinforcement Learning from Unit Test Feedbackcited by 110
  4. 2023Execution-based Code Generation using Deep Reinforcement Learningcited by 95
  5. 2021CVEfixes: Automated Collection of Vulnerabilities and Their Fixes from Open-Source Softwarecited by 89
  6. 2024Instruction Tuning for Secure Code Generationcited by 63
  7. 2024How secure is AI-generated Code: A Large-Scale Comparison of Large Language Modelscited by 23
  8. 2024Comparison of Static Application Security Testing Tools and Large Language Models for Repo-level Vulnerability Detectioncited by 18
  9. 2024LLM-Powered Code Vulnerability Repair with Reinforcement Learning and Semantic Rewardcited by 16
  10. 2024DLAP: A Deep Learning Augmented Large Language Model Prompting Framework for Software Vulnerability Detectioncited by 13
  11. 2025Guiding AI to Fix Its Own Flaws: An Empirical Study on LLM-Driven Secure Code Generationcited by 12
  12. 2024Code Security Vulnerability Repair Using Reinforcement Learning with Large Language Modelscited by 10
  13. 2025Large Language Model Guided Self-Debugging Code Generationcited by 7
  14. 2025Pool Me Wisely: On the Effect of Pooling in Transformer-Based Modelscited by 6
  15. 2025The Debugging Decay Index: Rethinking Debugging Strategies for Code LLMscited by 5
  16. 2026Secure Code Generation via Online Reinforcement Learning with Vulnerability Reward Modelcited by 4
  17. 2024Unleashing Artificial Cognition: Integrating Multiple AI Systemscited by 3
  18. 2025R+R: Security Vulnerability Dataset Quality Is Criticalcited by 3
  19. 2025An Empirical Study of Vulnerabilities in Python Packages and Their Detectioncited by 2
  20. 2026Adaptive Hierarchical Evaluation of LLMs and SAST tools for CWE Prediction in Pythoncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Out of Distribution, Out of Luck: How Well Can LLMs Trained on Vulnerability Datasets Detect Top 25 CWE Weaknesses?
  2. 2026Vul4Py: Benchmarking Automated Vulnerability Repair in Python with Paired Exploit and Functional Oracles
  3. 2026Test-Time Training Undermines Safety Guardrails
  4. 2026AuraForge: Scaling Security Supervision for Training Coding Agents
  5. 2026Learn from Your Mistakes: Tree-like Self-Play for Secure Code LLMs
  6. 2026From SFT to RL: Demystifying the Post-Training Pipeline for LLM-based Vulnerability Detection
  7. 2025Multi-level Certified Defense Against Poisoning Attacks in Offline Reinforcement Learning
  8. 2024Evaluating Defences against Unsafe Feedback in RLHF