# SecureNet: A Comparative Study of DeBERTa and Large Language Models for Phishing Detection

arXiv: https://arxiv.org/abs/2406.06663 · HTML: https://mlsanity.com/paper/2406.06663/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 41 references that are in the map.
- First posted on arXiv 10 Jun 2024.

## Builds on (most cited first, 17 of 17)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [Multi-step Jailbreaking Privacy Attacks on ChatGPT](https://mlsanity.com/paper/2304.05197/) (2023)
- [Pre-Trained Models: Past, Present and Future](https://mlsanity.com/paper/2106.07139/) (2021)
- [Recurrent Neural Networks (RNNs): A gentle Introduction and Overview](https://mlsanity.com/paper/1912.05911/) (2019)
- [ChatSpamDetector: Leveraging Large Language Models for Effective Phishing Email Detection](https://mlsanity.com/paper/2402.18093/) (2024)
- [From Chatbots to PhishBots? -- Preventing Phishing scams created using ChatGPT, Google Bard and Claude](https://mlsanity.com/paper/2310.19181/) (2023)
- [Detecting Scams Using Large Language Models](https://mlsanity.com/paper/2402.03147/) (2024)
- [An Explainable Transformer-based Model for Phishing Email Detection: A Large Language Model Approach](https://mlsanity.com/paper/2402.13871/) (2024)
- [Prompted Contextual Vectors for Spear-Phishing Detection](https://mlsanity.com/paper/2402.08309/) (2024)
- [A Deep Learning Model with Hierarchical LSTMs and Supervised Attention for Anti-Phishing](https://mlsanity.com/paper/1805.01554/) (2018)
- [Deep Reinforcement Learning for Detecting Malicious Websites](https://mlsanity.com/paper/1905.09207/) (2019)
- [A framework for securing email entrances and mitigating phishing impersonation attacks](https://mlsanity.com/paper/2312.04100/) (2023)
- [Classifying spam emails using agglomerative hierarchical clustering and a topic-based approach](https://mlsanity.com/paper/2402.05296/) (2024)

## Built on by (most cited first, 2 of 2)

- [E-PhishGen: Unlocking Novel Research in Phishing Email Detection](https://mlsanity.com/paper/2509.01791/) (2025)
- [The Phish, The Spam, and The Valid: Generating Feature-Rich Emails for Benchmarking LLMs](https://mlsanity.com/paper/2511.21448/) (2025)

## Abstract (arXiv)

Phishing, whether through email, SMS, or malicious websites, poses a major threat to organizations by using social engineering to trick users into revealing sensitive information. It not only compromises company's data security but also incurs significant financial losses. In this paper, we investigate whether the remarkable performance of Large Language Models (LLMs) can be leveraged for particular task like text classification, particularly detecting malicious content and compare its results with state-of-the-art Deberta V3 (DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing) model. We systematically assess the potential and limitations of both approaches using comprehensive public datasets comprising diverse data sources such as email, HTML, URL, SMS, and synthetic data generation. Additionally, we demonstrate how LLMs can generate convincing phishing emails, making it harder to spot scams and evaluate the performance of both models in this context. Our study delves further into the challenges encountered by DeBERTa V3 during its training phases, fine-tuning methodology and transfer learning processes. Similarly, we examine the challenges associated with LLMs and assess their respective performance. Among our experimental approaches, the transformer-based DeBERTa method emerged as the most effective, achieving a test dataset (HuggingFace phishing dataset) recall (sensitivity) of 95.17% closely followed by GPT-4 providing a recall of 91.04%. We performed additional experiments with other datasets on the trained DeBERTa V3 model and LLMs like GPT 4 and Gemini 1.5. Based on our findings, we provide valuable insights into the effectiveness and robustness of these advanced language models, offering a detailed comparative analysis that can inform future research efforts in strengthening cybersecurity measures for detecting and mitigating phishing threats.