mlsanityData to 7 Oct 2026

CyberGym-E2E: Scalable Real-World Benchmark for AI Agents' End-to-End Cybersecurity Capabilities

arXiv 2606.04460 ↗First posted 3 Jun 2026cs.CR cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

AI has the potential to transform cybersecurity by enabling systems that can autonomously detect, analyze, and remediate software vulnerabilities. However, existing cybersecurity evaluations of AI systems are limited in scale or scope, and fail to capture the end-to-end lifecycle of real-world software vulnerability discovery and remediation. To address this gap, we propose CyberGym-E2E, a large-scale and realistic end-to-end cybersecurity benchmark that comprehensively evaluates AI agents' abilities across the full lifecycle of vulnerability discovery, PoC generation, and patch generation. CyberGym-E2E is comprehensive and scalable, as we build an automated, agent-enhanced pipeline for transforming open-source vulnerability data into realistic evaluation environments. Currently, the benchmark consists of 920 real-world vulnerabilities across 139 different open-source projects.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 10 of its 20 references that are in the map. First posted on arXiv 3 Jun 2026.

Builds on

10 of 20 references.

References · 10

  1. 2024Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Modelscited by 156
  2. 2024Vulnerability Detection with Code Language Models: How Far Are We?cited by 133
  3. 2024NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Securitycited by 90
  4. 2025CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilitiescited by 90
  5. 2025CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scalecited by 71
  6. 2025SEC-bench: Automated Benchmarking of LLM Agents on Real-World Software Security Taskscited by 69
  7. 2025CWEval: Outcome-driven Evaluation on Functionality and Security of LLM Code Generationcited by 55
  8. 2025BountyBench: Dollar Impact of AI Agent Attackers and Defenders on Real-World Cybersecurity Systemscited by 37
  9. 2025Frontier AI's Impact on the Cybersecurity Landscapecited by 21
  10. 2025SecRepoBench: Benchmarking Code Agents for Secure Code Completion in Real-World Repositoriescited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Red-Teaming the Agentic Red-Teamcited by 0
  2. 2026CyberFactory: Scaling Cyber Security Capabilities with Instances from the Wildcited by 0
  3. 2026Vulnerability Localization Benchmark: Measuring Agentic Security Analysis at Repository Scalecited by 0
  4. 2026MobileCybench: Evaluating Agent Vulnerability Discovery via Executable Probescited by 0
  5. 2026SecProbe: Adaptive Evaluation of Coding Agents on Cybersecurity Vulnerabilitiescited by 0
  6. 2026AuraForge: Scaling Security Supervision for Training Coding Agentscited by 0