# Generative Artificial Intelligence-Supported Pentesting: A Comparison between Claude Opus, GPT-4, and Copilot

arXiv: https://arxiv.org/abs/2501.06963 · HTML: https://mlsanity.com/paper/2501.06963/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 35 references that are in the map.
- First posted on arXiv 12 Jan 2025.

## Builds on (most cited first, 4 of 4)

- [From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy](https://mlsanity.com/paper/2307.00691/) (2023)
- [Getting pwn'd by AI: Penetration Testing with Large Language Models](https://mlsanity.com/paper/2308.00121/) (2023)
- [The Social Impact of Generative AI: An Analysis on ChatGPT](https://mlsanity.com/paper/2403.04667/) (2024)
- [Exploring ChatGPT and its Impact on Society](https://mlsanity.com/paper/2403.14643/) (2024)

## Built on by (most cited first, 2 of 2)

- [To Patch or Not to Patch: Motivations, Challenges, and Implications for Cybersecurity](https://mlsanity.com/paper/2502.17703/) (2025)
- [Evaluating the Ability of Large Language Models to Identify Adherence to CONSORT Reporting Guidelines in Randomized Controlled Trials: A Methodological Evaluation Study](https://mlsanity.com/paper/2511.13107/) (2025)

## Abstract (arXiv)

The advent of Generative Artificial Intelligence (GenAI) has brought a significant change to our society. GenAI can be applied across numerous fields, with particular relevance in cybersecurity. Among the various areas of application, its use in penetration testing (pentesting) or ethical hacking processes is of special interest. In this paper, we have analyzed the potential of leading generic-purpose GenAI tools-Claude Opus, GPT-4 from ChatGPT, and Copilot-in augmenting the penetration testing process as defined by the Penetration Testing Execution Standard (PTES). Our analysis involved evaluating each tool across all PTES phases within a controlled virtualized environment. The findings reveal that, while these tools cannot fully automate the pentesting process, they provide substantial support by enhancing efficiency and effectiveness in specific tasks. Notably, all tools demonstrated utility; however, Claude Opus consistently outperformed the others in our experimental scenarios.