# Detecting Phishing Sites Using ChatGPT

arXiv: https://arxiv.org/abs/2306.05816 · HTML: https://mlsanity.com/paper/2306.05816/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jun 2023.

## Built on by (most cited first, 9 of 9)

- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2312.02003/) (2023)
- [Large Language Models in Cybersecurity: State-of-the-Art](https://mlsanity.com/paper/2402.00891/) (2024)
- [A Survey of Large Language Models in Cybersecurity](https://mlsanity.com/paper/2402.16968/) (2024)
- [PhishLang: A Real-Time, Fully Client-Side Phishing Detection Framework Using MobileBERT](https://mlsanity.com/paper/2408.05667/) (2024)
- [Zero-Shot Spam Email Classification Using Pre-trained Large Language Models](https://mlsanity.com/paper/2405.15936/) (2024)
- [Gen-AI for User Safety: A Survey](https://mlsanity.com/paper/2411.06606/) (2024)
- [Construction and Evaluation of LLM-based agents for Semi-Autonomous penetration testing](https://mlsanity.com/paper/2502.15506/) (2025)
- [Eliciting Least-to-Most Reasoning for Phishing URL Detection](https://mlsanity.com/paper/2601.20270/) (2026)

## Abstract (arXiv)

The emergence of Large Language Models (LLMs), including ChatGPT, is having a significant impact on a wide range of fields. While LLMs have been extensively researched for tasks such as code generation and text synthesis, their application in detecting malicious web content, particularly phishing sites, has been largely unexplored. To combat the rising tide of cyber attacks due to the misuse of LLMs, it is important to automate detection by leveraging the advanced capabilities of LLMs. In this paper, we propose a novel system called ChatPhishDetector that utilizes LLMs to detect phishing sites. Our system involves leveraging a web crawler to gather information from websites, generating prompts for LLMs based on the crawled data, and then retrieving the detection results from the responses generated by the LLMs. The system enables us to detect multilingual phishing sites with high accuracy by identifying impersonated brands and social engineering techniques in the context of the entire website, without the need to train machine learning models. To evaluate the performance of our system, we conducted experiments on our own dataset and compared it with baseline systems and several LLMs. The experimental results using GPT-4V demonstrated outstanding performance, with a precision of 98.7% and a recall of 99.6%, outperforming the detection results of other LLMs and existing systems. These findings highlight the potential of LLMs for protecting users from online fraudulent activities and have important implications for enhancing cybersecurity measures.