mlsanityData to 14 Oct 2026

Cyber-Zero: Training Cybersecurity Agents without Runtime

arXiv 2508.00910 ↗First posted 29 Jul 2025cs.CR cs.CL cs.LGcited by 22

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Large Language Models (LLMs) have achieved remarkable success in software engineering tasks when trained with executable runtime environments, particularly in resolving GitHub issues. However, such runtime environments are often unavailable in other domains, especially cybersecurity, where challenge configurations and execution contexts are ephemeral or restricted. We present Cyber-Zero, the first runtime-free framework for synthesizing high-quality agent trajectories to train cybersecurity LLMs. Cyber-Zero leverages publicly available CTF writeups and employs persona-driven LLM simulation to reverse-engineer runtime behaviors and generate realistic, long-horizon interaction sequences without actual environments. Using trajectories synthesized by Cyber-Zero, we train LLM-based agents that achieve up to 13.1% absolute performance gains over baseline models on three prominent CTF benchmarks: InterCode-CTF, NYU CTF Bench, and Cybench. Our best model, Cyber-Zero-32B, establishes new state-of-the-art performance among open-weight models, matching the capabilities of proprietary systems like DeepSeek-V3-0324 and Claude-3.5-Sonnet while offering superior cost-effectiveness, and demonstrating that runtime-free trajectory synthesis can effectively democratize the development of state-of-the-art cybersecurity agents.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 28 of its 53 references that are in the map. First posted on arXiv 29 Jul 2025.

Builds on

28 of 53 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025SecRepoBench: Benchmarking Code Agents for Secure Code Completion in Real-World Repositories29 Apr
  2. 2024LLM Agents can Autonomously Hack Websites6 Feb

Other references · 26

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2024OpenAI o1 System Cardcited by 1,244
  6. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  7. 2023StarCoder: may the source be with you!cited by 799
  8. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 502
  9. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 417
  10. 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 336
  11. 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 285
  12. 2025SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolutioncited by 263
14 more
  1. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 235
  2. 2023Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Modelscited by 138
  3. 2024Evaluating Frontier Models for Dangerous Capabilitiescited by 128
  4. 2024An Adversarial Perspective on Machine Unlearning for AI Safetycited by 123
  5. 2025SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolutioncited by 90
  6. 2024Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvementcited by 61
  7. 2024CYBERSECEVAL 3: Advancing the Evaluation of Cybersecurity Risks and Capabilities in Large Language Modelscited by 58
  8. 2024On Evaluating the Durability of Safeguards for Open-Weight LLMscited by 52
  9. 2023SecQA: A Concise Question-Answering Dataset for Evaluating Large Language Models in Computer Securitycited by 37
  10. 2023From Text to MITRE Techniques: Exploring the Malicious Use of Large Language Models for Generating Cyber Attack Payloadscited by 32
  11. 2024Astraios: Parameter-Efficient Instruction Tuning Code Large Language Modelscited by 23
  12. 2025AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defensescited by 15
  13. 2025CAI: An Open, Bug Bounty-Ready Cybersecurity AIcited by 14
  14. 2024SeCodePLT: A Unified Platform for Evaluating the Security of Code GenAIcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 17.

22 papers.

Sort:

By area:

  1. 2026Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?cited by 18
  2. 2025Training Language Model Agents to Find Vulnerabilities with CTF-Dojocited by 15
  3. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 15
  4. 2025Uncovering Vulnerabilities of LLM-Assisted Cyber Threat Intelligencecited by 6
  5. 2026Watermarking LLM Agent Trajectoriescited by 4
  6. 2025HackWorld: Evaluating Computer-Use Agents on Exploiting Web Application Vulnerabilitiescited by 2
  7. 2026AgentSnare: Learning to Delay, Divert, and Defuse Autonomous Penetration Agentscited by 2
  8. 2026PenForge: On-the-Fly Expert Agent Construction for Automated Penetration Testingcited by 1
  9. 2026To Defend Against Cyber Attacks, We Must Teach AI Agents to Hackcited by 1
  10. 2026Towards a Neural Debugger for Pythoncited by 1
  11. 2026AI In Cybersecurity Education -- Scalable Agentic CTF Design Principles and Educational Outcomescited by 1
  12. 2026Challenges and Future Directions in Agentic Reverse Engineering Systemscited by 1
10 more
  1. 2026Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoringcited by 1
  2. 2026Trident : How to Break Deep Reinforcement Learning Cyber Defenses (Agentic)cited by 1
  3. 2026RangeFactory: Scalable Construction of Multi-Hop Cyber Rangescited by 1
  4. 2025Coherence-driven inference for cybersecuritycited by 0
  5. 2026XekRung Technical Reportcited by 0
  6. 2026CyberEvolver: Structured Self-Evolution for Cybersecurity Agents On the Flycited by 0
  7. 2026Mastermind: Strategy-grounded Learning for Repository-Scale Vulnerability Reproductioncited by 0
  8. 2026CyberForge: Verified Vulnerability Injection at Repository Level for Cybersecurity Agent Trainingcited by 0
  9. 2026CyberFactory: Scaling Cyber Security Capabilities with Instances from the Wildcited by 0
  10. 2026Staying on the Attack Path: Structured State for Long-Horizon Automated Penetration Testingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Out of the Cage: How Stochastic Parrots Win in Cyber Security Environments
  2. 2026ZeroDayBench: Evaluating LLM Agents on Unseen Zero-Day Vulnerabilities for Cyberdefense
  3. 2026Systematic Capability Benchmarking of Frontier Large Language Models for Offensive Cyber Tasks
  4. 2026Autonomous LLM Agents & CTFs: A Second Look
  5. 2025AgentCyTE: Leveraging Agentic AI to Generate Cybersecurity Training & Experimentation Scenarios
  6. 2025Less Data, More Security: Advancing Cybersecurity LLMs Specialization via Resource-Efficient Domain-Adaptive Continuous Pre-training with Minimal Tokens
  7. 2026Capture the Flags: Family-Based Evaluation of Agentic LLMs via Semantics-Preserving Transformations
  8. 2026Cyber Defense Benchmark: Agentic Threat Hunting Evaluation for LLMs in SecOps