mlsanityData to 7 Oct 2026

CS-Guard: Benchmarking LLM Guardrails for Code Generation Security

arXiv 2609.09798 ↗First posted 9 Sep 2026cs.CR cs.AIcited by 0

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Large language models (LLMs) have been ex- ploited to generate malware, but the effective- ness of guardrails for code generation secu- rity remains unclear. We introduce CS-Guard, the first benchmark to systematically evalu- ate guardrails for code generation security. It covers 1) text-to-code generation with 1000 high-quality malware-generation prompts, 7 jailbreak attacks, and a novel fictional scenario attack (FSA) that embeds malicious intent in a legitimate fictional software-development sce- nario; and 2) code-to-code generation with 331 code prompts spanning code infilling, code completion, and code translation. We empiri- cally evaluate 9 guardrails across seven LLMs. We find that current guardrails perform poorly against malicious code-generation re- quests: for text-to-code, the average attack success rate (ASR) after jailbreaks reaches about 50% for many guardrails; for code-to- code, average ASR approaches 100% on base LLMs and remains high across many guardrails (14.4% to nearly 100%). Our FSA also achieves ASR close to 100% across many guardrails, raising major reliability concerns for real-world software development. To sup- port future research, CS-Guard uses a modular three-layer guardrail taxonomy that lets devel- opers register guardrails for evaluation. We release the benchmark and data to enable fur- ther community evaluation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 42 of its 62 references that are in the map. First posted on arXiv 9 Sep 2026.

Builds on

42 of 62 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025Active Honeypot Guardrail System: Probing and Confirming Multi-Turn LLM Jailbreaks16 Oct
  2. 2025LLMs Caught in the Crossfire: Malware Requests and Jailbreak Challenges9 Jun
  3. 2025PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attacks20 May
  4. 2025Smoke and Mirrors: Jailbreaking LLM-based Code Generation via Implicit Malicious Prompts23 Mar
  5. 2024Emoji Attack: Enhancing Jailbreak Attacks Against Judge LLM Detection1 Nov
  6. 2024RMCBench: Benchmarking Large Language Models' Resistance to Malicious Code23 Sep
  7. 2024Protecting Your LLMs with Information Bottleneck22 Apr
  8. 2024Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapes1 Mar
  9. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding14 Feb
  10. 2024Fight Back Against Jailbreaking via Prompt Adversarial Tuning9 Feb
  11. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks30 Jan
  12. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
4 more
  1. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  2. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  3. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 26

  1. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  2. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  3. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  4. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  5. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
  6. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 417
  7. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  8. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  9. 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 149
  10. 2022Large Language Models Meet NL2Code: A Surveycited by 135
  11. 2023Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Modelscited by 130
  12. 2023InferFix: End-to-End Program Repair with LLMscited by 113
14 more
  1. 2024RedCode: Risky Code Execution and Generation Benchmark for Code Agentscited by 87
  2. 2023On the Robustness of Code Generation Techniques: An Empirical Study on GitHub Copilotcited by 64
  3. 2022Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?cited by 63
  4. 2024Uncovering Safety Risks of Large Language Models through Concept Activation Vectorcited by 63
  5. 2022CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Modelscited by 48
  6. 2025Improving LLM Safety Alignment with Dual-Objective Optimizationcited by 40
  7. 2023SALLM: Security Assessment of Generated Codecited by 35
  8. 2023DeceptPrompt: Exploiting LLM-driven Code Generation via Adversarial Natural Language Instructionscited by 31
  9. 2024Malla: Demystifying Real-world Large Language Model Integrated Malicious Servicescited by 24
  10. 2025Devstral: Fine-tuning Language Models for Coding Agent Applicationscited by 23
  11. 2025Poly-Guard: Massive Multi-Domain Safety Policy-Grounded Guardrail Datasetcited by 14
  12. 2024Black-Box Adversarial Attacks on LLM-Based Code Completioncited by 13
  13. 2025MOCHA: Are Code Language Models Robust Against Multi-Turn Malicious Coding Prompts?cited by 5
  14. 2026Large Language Models (LLMs) and Generative AI in Cybersecurity and Privacy: A Survey of Dual-Use Risks, AI-Generated Malware, Explainability, and Defensive Strategiescited by 2

Built on by

Not cited yet.