# Give LLMs a Security Course: Securing Retrieval-Augmented Code Generation via Knowledge Injection

arXiv: https://arxiv.org/abs/2504.16429 · HTML: https://mlsanity.com/paper/2504.16429/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 29 of its 54 references that are in the map.
- First posted on arXiv 23 Apr 2025.

## Placed in

- Retrieval-augmented generation / [Privacy and security of RAG](https://mlsanity.com/rag/privacy-and-security-of-rag/)

## Builds on (most cited first, 29 of 29)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Efficient Streaming Language Models with Attention Sinks](https://mlsanity.com/paper/2309.17453/) (2023)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019)
- [CodeBLEU: a Method for Automatic Evaluation of Code Synthesis](https://mlsanity.com/paper/2009.10297/) (2020)
- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [Language Models can Solve Computer Tasks](https://mlsanity.com/paper/2303.17491/) (2023)
- [From Sparse to Soft Mixtures of Experts](https://mlsanity.com/paper/2308.00951/) (2023)
- [Large Language Models for Code: Security Hardening and Adversarial Testing](https://mlsanity.com/paper/2302.05319/) (2023)
- [jina-embeddings-v3: Multilingual Embeddings With Task LoRA](https://mlsanity.com/paper/2409.10173/) (2024)
- [Structured Chain-of-Thought Prompting for Code Generation](https://mlsanity.com/paper/2305.06599/) (2023)
- [An Empirical Study of the Non-determinism of ChatGPT in Code Generation](https://mlsanity.com/paper/2308.02828/) (2023)
- [Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documents](https://mlsanity.com/paper/2310.19923/) (2023)
- [ReposVul: A Repository-Level High-Quality Vulnerability Dataset](https://mlsanity.com/paper/2401.13169/) (2024)
- [Prompting Techniques for Secure Code Generation: A Systematic Investigation](https://mlsanity.com/paper/2407.07064/) (2024)
- [Using AI Assistants in Software Development: A Qualitative Study on Security Practices and Concerns](https://mlsanity.com/paper/2405.06371/) (2024)
- [How secure is AI-generated Code: A Large-Scale Comparison of Large Language Models](https://mlsanity.com/paper/2404.18353/) (2024)
- [EVOR: Evolving Retrieval for Code Generation](https://mlsanity.com/paper/2402.12317/) (2024)
- [LLM Security Guard for Code](https://mlsanity.com/paper/2405.01103/) (2024)
- [Preference-Guided Refactored Tuning for Retrieval Augmented Code Generation](https://mlsanity.com/paper/2409.15895/) (2024)
- [Exploring the Security Threats of Knowledge Base Poisoning in Retrieval-Augmented Code Generation](https://mlsanity.com/paper/2502.03233/) (2025)
- [SOSecure: Safer Code Generation with RAG and StackOverflow Discussions](https://mlsanity.com/paper/2503.13654/) (2025)
- [Rethinking CyberSecEval: An LLM-Aided Approach to Evaluation Critique](https://mlsanity.com/paper/2411.08813/) (2024)

## Built on by (most cited first, 16 of 16)

- [AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities](https://mlsanity.com/paper/2508.11126/) (2025)
- [Retrieval-Augmented Code Generation: A Survey with Focus on Repository-Level Approaches](https://mlsanity.com/paper/2510.04905/) (2025)
- [RESCUE: Retrieval Augmented Secure Code Generation](https://mlsanity.com/paper/2510.18204/) (2025)
- [Confundo: Learning to Generate Robust Poison for Practical RAG Systems](https://mlsanity.com/paper/2602.06616/) (2026)
- [Enhancing Reliability in LLM-Based Secure Code Generation](https://mlsanity.com/paper/2605.24300/) (2026)
- [Agent-Assisted Side-Channel Attacks on Non-Prefix KV Cache in RAG](https://mlsanity.com/paper/2606.21842/) (2026)
- [Exploring the Security Threats of Retriever Backdoors in Retrieval-Augmented Code Generation](https://mlsanity.com/paper/2512.21681/) (2025)
- [Knowdit: Agentic Smart Contract Vulnerability Detection with Auditing Knowledge Summarization](https://mlsanity.com/paper/2603.26270/) (2026)
- [Correct Code, Vulnerable Dependencies: A Large Scale Measurement Study of LLM-Specified Library Versions](https://mlsanity.com/paper/2605.06279/) (2026)
- [CodePoisonRAG: Knowledge Poisoning Attacks on Retrieval-Augmented Code Generation](https://mlsanity.com/paper/2609.02774/) (2026)
- [A Quasi-Experimental Developer Study of Security Training in LLM-Assisted Web Application Development](https://mlsanity.com/paper/2604.17763/) (2026)
- [Activating Latent Security Knowledge through LLM-Guided Risk Analysis for Secure Code Generation](https://mlsanity.com/paper/2606.16244/) (2026)
- [From Adoption to Deployment: A Qualitative Study on AI Integration in Software Development Practice](https://mlsanity.com/paper/2607.16660/) (2026)
- [MACGen: Toward Functionally Correct and Secure Code Generation via Multi-Agent Collaboration](https://mlsanity.com/paper/2608.25457/) (2026)
- [Interpreting and Steering for Safe and Correct Code Generation](https://mlsanity.com/paper/2608.30025/) (2026)
- [SEEK: Agentic Knowledge Graph Retrieval for Secure Code Generation](https://mlsanity.com/paper/2610.14968/) (2026)

## Abstract (arXiv)

Retrieval-Augmented Code Generation (RACG) leverages external knowledge to enhance Large Language Models (LLMs) in code synthesis, improving the functional correctness of the generated code. However, existing RACG systems largely overlook security, leading to substantial risks. Especially, the poisoning of malicious code into knowledge bases can mislead LLMs, resulting in the generation of insecure outputs, which poses a critical threat in modern software development. To address this, we propose a security-hardening framework for RACG systems, CodeGuarder, that shifts the paradigm from retrieving only functional code examples to incorporating both functional code and security knowledge. Our framework constructs a security knowledge base from real-world vulnerability databases, including secure code samples and root cause annotations. For each code generation query, a retriever decomposes the query into fine-grained sub-tasks and fetches relevant security knowledge. To prioritize critical security guidance, we introduce a re-ranking and filtering mechanism by leveraging the LLMs' susceptibility to different vulnerability types. This filtered security knowledge is seamlessly integrated into the generation prompt. Our evaluation shows CodeGuarder significantly improves code security rates across various LLMs, achieving average improvements of 20.12% in standard RACG, and 31.53% and 21.91% under two distinct poisoning scenarios without compromising functional correctness. Furthermore, CodeGuarder demonstrates strong generalization, enhancing security even when the targeted language's security knowledge is lacking. This work presents CodeGuarder as a pivotal advancement towards building secure and trustworthy RACG systems.