mlsanityData to 14 Oct 2026

Dynamic Risk Assessments for Offensive Cybersecurity Agents

arXiv 2505.18384 ↗First posted 23 May 2025cs.CR cs.AIcited by 8

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Foundation models are increasingly becoming better autonomous programmers, raising the prospect that they could also automate dangerous offensive cyber-operations. Current frontier model audits probe the cybersecurity risks of such agents, but most fail to account for the degrees of freedom available to adversaries in the real world. In particular, with strong verifiers and financial incentives, agents for offensive cybersecurity are amenable to iterative improvement by would-be adversaries. We argue that assessments should take into account an expanded threat model in the context of cybersecurity, emphasizing the varying degrees of freedom that an adversary may possess in stateful and non-stateful environments within a fixed compute budget. We show that even with a relatively small compute budget (8 H100 GPU Hours in our study), adversaries can improve an agent's cybersecurity capability on InterCode CTF by more than 40% relative to the baseline -- without any external assistance. These results highlight the need to evaluate agents' cybersecurity risk in a dynamic manner, painting a more representative picture of risk.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 60 of its 83 references that are in the map. First posted on arXiv 23 May 2025.

Builds on

60 of 83 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Hacking CTFs with Plain Agents3 Dec
  2. 2024Teams of LLM Agents can Exploit Zero-Day Vulnerabilities2 Jun
  3. 2024LLM Agents can Autonomously Exploit One-day Vulnerabilities11 Apr
  4. 2024LLM Agents can Autonomously Hack Websites6 Feb

Other references · 56

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  6. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  7. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  8. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  9. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  10. 2020Learning to summarize from human feedbackcited by 2,588
  11. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  12. 2025Gemma 3 Technical Reportcited by 1,824
38 more
  1. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  2. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  3. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  4. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  5. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  6. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  7. 2022Large Language Models Can Self-Improvecited by 664
  8. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  9. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
  10. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 480
  11. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
  12. 2024Automated Design of Agentic Systemscited by 407
  13. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 349
  14. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 344
  15. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Trainingcited by 295
  16. 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 238
  17. 2024V-STaR: Training Verifiers for Self-Taught Reasonerscited by 217
  18. 2024Recursive Introspection: Teaching Language Model Agents How to Self-Improvecited by 165
  19. 2024Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Modelscited by 164
  20. 2023Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Modelscited by 138
  21. 2024An Adversarial Perspective on Machine Unlearning for AI Safetycited by 123
  22. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 120
  23. 2023PentestGPT: An LLM-empowered Automatic Penetration Testing Toolcited by 97
  24. 2024AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attackscited by 96
  25. 2024NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Securitycited by 93
  26. 2024CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Modelscited by 89
  27. 2023Getting pwn'd by AI: Penetration Testing with Large Language Modelscited by 83
  28. 2024Planning In Natural Language Improves LLM Search For Code Generationcited by 83
  29. 2023ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agentcited by 81
  30. 2024On the Societal Impact of Open Foundation Modelscited by 65
  31. 2024One fish, two fish, but not the whole sea: Alignment reduces language models' conceptual diversitycited by 58
  32. 2024CyberMetric: A Benchmark Dataset based on Retrieval-Augmented Generation for Evaluating LLMs in Cybersecurity Knowledgecited by 55
  33. 2024On Evaluating the Durability of Safeguards for Open-Weight LLMscited by 52
  34. 2024The Larger the Better? Improved LLM Code-Generation via Budget Reallocationcited by 43
  35. 2023SecQA: A Concise Question-Answering Dataset for Evaluating Large Language Models in Computer Securitycited by 37
  36. 2024An Empirical Evaluation of LLMs for Solving Offensive Security Challengescited by 35
  37. 2024PenHeal: A Two-Stage LLM Framework for Automated Pentesting and Optimal Remediationcited by 30
  38. 2023Using Large Language Models for Cybersecurity Capture-The-Flag Challenges and Certification Questionscited by 23

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 2.

8 papers.

  1. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 170
  2. 2025Your Agent May Misevolve: Emergent Risks in Self-evolving LLM Agentscited by 62
  3. 2025Accumulating Context Changes the Beliefs of Language Modelscited by 28
  4. 2025PoCo: Agentic Proof-of-Concept Exploit Generation for Smart Contractscited by 8
  5. 2025Large Language Models Develop Novel Social Biases Through Adaptive Explorationcited by 2
  6. 2026AI Agents Enable Adaptive Computer Wormscited by 2
  7. 2025Best Practices for Biorisk Evaluations on Open-Weight Bio-Foundation Modelscited by 1
  8. 2026EvoRiskBench: An Evolving Benchmark for Runtime Security Risks in Workspace Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026To Defend Against Cyber Attacks, We Must Teach AI Agents to Hack
  2. 2025Cybersecurity AI: Evaluating Agentic Cybersecurity in Attack/Defense CTFs
  3. 2026Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response
  4. 2026Beyond Success Rate: Cost-Aware Evaluation of Offensive and Defensive Security Agents
  5. 2026A Survey of Agentic AI and Cybersecurity: Challenges, Opportunities and Use-case Prototypes
  6. 2026SecProbe: Adaptive Evaluation of Coding Agents on Cybersecurity Vulnerabilities
  7. 2026A New Framework for Cybersecurity Refusals in AI Agents
  8. 2026The Ethics of Autonomous AI Agents for Offensive Security