mlsanityData to 7 Oct 2026

Frontier AI's Impact on the Cybersecurity Landscape

arXiv 2504.05408 ↗First posted 7 Apr 2025cs.CR cs.AI cs.CYcited by 21

Not on the map. Why?

Abstract (arXiv)

The impact of frontier AI (i.e., AI agents and foundation models) in cybersecurity is rapidly increasing. In this paper, we comprehensively analyze this trend through multiple aspects: quantitative benchmarks, qualitative literature review, empirical evaluation, and expert survey. Our analyses consistently show that AI's capabilities and applications in attacks have exceeded those on the defensive side. Our empirical evaluation of widely used agent systems on cybersecurity benchmarks highlights that current AI agents struggle with flexible workflow planning and using domain-specific tools for complex security analysis -- capabilities particularly critical for defensive applications. Our expert survey of AI and security researchers and practitioners indicates a prevailing view that AI will continue to benefit attackers over defenders, though the gap is expected to narrow over time. These results show the urgent need to evaluate and mitigate frontier AI's risks, steering it towards benefiting cyber defenses. Responding to this need, we provide concrete calls to action regarding: the construction of new cybersecurity benchmarks, the development of AI agents for defense, the design of provably secure AI agents, the improvement of pre-deployment security testing and transparency, and the strengthening of user-oriented education and defenses. Our paper summary and blog are available at https://rdi.berkeley.edu/frontier-ai-impact-on-cybersecurity/.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 173 of its 382 references that are in the map. First posted on arXiv 7 Apr 2025.

Builds on

173 of 382 references.

References · 173

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  8. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  9. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  10. 2024GPT-4o System Cardcited by 4,467
  11. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  12. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
38 more
  1. 2020Extracting Training Data from Large Language Modelscited by 2,144
  2. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  3. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  4. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  5. 2024A Survey on LLM-as-a-Judgecited by 1,384
  6. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  7. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  8. 2025s1: Simple test-time scalingcited by 1,311
  9. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  10. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  11. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  12. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  13. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
  14. 2023Large Language Modelscited by 774
  15. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 739
  16. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  17. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  18. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  19. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  20. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 417
  21. 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 400
  22. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 389
  23. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  24. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
  25. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 328
  26. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
  27. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
  28. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 280
  29. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  30. 2019Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networkscited by 266
  31. 2023Certifying LLM Safety against Adversarial Promptingcited by 248
  32. 2024RepairAgent: An Autonomous, LLM-Based Agent for Program Repaircited by 225
  33. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  34. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attackscited by 208
  35. 2024SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?cited by 196
  36. 2018Formal Security Analysis of Neural Networks using Symbolic Intervalscited by 185
  37. 2024Agent AI: Surveying the Horizons of Multimodal Interactioncited by 167
  38. 2023Large-scale Multi-Modal Pre-trained Models: A Comprehensive Surveycited by 153

50 most cited of 173 All 173 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 12; 2026: 9.

21 papers.

Sort:

By area:

  1. 2025PromptArmor: Simple yet Effective Prompt Injection Defensescited by 98
  2. 2025CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scalecited by 71
  3. 2025Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agentscited by 66
  4. 2025BountyBench: Dollar Impact of AI Agent Attackers and Defenders on Real-World Cybersecurity Systemscited by 37
  5. 2025From CVE Entries to Verifiable Exploits: An Automated Multi-Agent Framework for Reproducing CVEscited by 17
  6. 2025RepIt: Steering Language Models with Concept-Specific Refusal Vectorscited by 10
  7. 2026CyberGym-E2E: Scalable Real-World Benchmark for AI Agents' End-to-End Cybersecurity Capabilitiescited by 6
  8. 2025Agentic Discovery and Validation of Android App Vulnerabilitiescited by 4
  9. 2026OpenSage: Self-programming Agent Generation Enginecited by 4
  10. 2025CyberSleuth: Autonomous Blue-Team LLM Agent for Web Attack Forensicscited by 3
  11. 2026The End of Code Review: Coding Agents Supersede Human Inspectioncited by 3
  12. 2025HackWorld: Evaluating Computer-Use Agents on Exploiting Web Application Vulnerabilitiescited by 2
9 more
  1. 2026Human Society-Inspired Approaches to Agentic AI Security: The 4C Frameworkcited by 2
  2. 2026Cybersecurity is the True Frontier for Generative AI Success or Failurecited by 2
  3. 2025RRTL: Red Teaming Reasoning Large Language Models in Tool Learningcited by 1
  4. 2025Unified Threat Detection and Mitigation Framework (UTDMF): Combating Prompt Injection, Deception, and Bias in Enterprise-Scale Transformerscited by 1
  5. 2025Cybersecurity AI: Evaluating Agentic Cybersecurity in Attack/Defense CTFscited by 1
  6. 2026AEGIS: White-Box Attack Path Generation using LLMs and Training Effectiveness Evaluation for Large-Scale Cyber Defence Exercisescited by 1
  7. 2026To Defend Against Cyber Attacks, We Must Teach AI Agents to Hackcited by 1
  8. 2026Toward Scalable Automated Repository-Level Datasets for Software Vulnerability Detectioncited by 0
  9. 2026CyberForge: Verified Vulnerability Injection at Repository Level for Cybersecurity Agent Trainingcited by 0