# DeepGuard: Secure Code Generation via Multi-Layer Semantic Aggregation

arXiv: https://arxiv.org/abs/2604.09089 · HTML: https://mlsanity.com/paper/2604.09089/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 10 Apr 2026.

## Builds on (most cited first, 14 of 14)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [Large Language Models for Code: Security Hardening and Adversarial Testing](https://mlsanity.com/paper/2302.05319/) (2023)
- [How Secure is Code Generated by ChatGPT?](https://mlsanity.com/paper/2304.09655/) (2023)
- [Instruction Tuning for Secure Code Generation](https://mlsanity.com/paper/2402.09497/) (2024)
- [What Do They Capture? -- A Structural Analysis of Pre-Trained Language Models for Source Code](https://mlsanity.com/paper/2202.06840/) (2022)
- [Constrained Decoding for Secure Code Generation](https://mlsanity.com/paper/2405.00218/) (2024)
- [Assessing the Security of GitHub Copilot Generated Code -- A Targeted Replication Study](https://mlsanity.com/paper/2311.11177/) (2023)
- [Can We Trust Large Language Models Generated Code? A Framework for In-Context Learning, Security Patterns, and Code Evaluations Across Diverse LLMs](https://mlsanity.com/paper/2406.12513/) (2024)
- [A User-centered Security Evaluation of Copilot](https://mlsanity.com/paper/2308.06587/) (2023)
- [SecCoder: Towards Generalizable and Robust Secure Code Generation](https://mlsanity.com/paper/2410.01488/) (2024)
- [Guiding AI to Fix Its Own Flaws: An Empirical Study on LLM-Driven Secure Code Generation](https://mlsanity.com/paper/2506.23034/) (2025)
- [Unveiling Code Pre-Trained Models: Investigating Syntax and Semantics Capacities](https://mlsanity.com/paper/2212.10017/) (2022)

## Built on by (most cited first, 2 of 2)

- [Functional and Secure Code Generation with Task Vectors](https://mlsanity.com/paper/2607.07881/) (2026)
- [MACGen: Toward Functionally Correct and Secure Code Generation via Multi-Agent Collaboration](https://mlsanity.com/paper/2608.25457/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) for code generation can replicate insecure patterns from their training data. To mitigate this, a common strategy for security hardening is to fine-tune models using supervision derived from the final transformer layer. However, this design may suffer from a final-layer bottleneck: vulnerability-discriminative cues can be distributed across layers and become less detectable near the output representations optimized for next-token prediction. To diagnose this issue, we perform layer-wise linear probing. We observe that vulnerability-related signals are most detectable in a band of intermediate-to-upper layers yet attenuate toward the final layers. Motivated by this observation, we introduce DeepGuard, a framework that leverages distributed security-relevant cues by aggregating representations from multiple upper layers via an attention-based module. The aggregated signal powers a dedicated security analyzer within a multi-objective training objective that balances security enhancement and functional correctness, and further supports a lightweight inference-time steering strategy. Extensive experiments across five code LLMs demonstrate that DeepGuard improves the secure-and-correct generation rate by an average of 11.9% over strong baselines such as SVEN. It also preserves functional correctness while exhibiting generalization to held-out vulnerability types. Our code is public at https://github.com/unknownhl/DeepGuard.