# An Improved Transformer-based Model for Detecting Phishing, Spam, and Ham: A Large Language Model Approach

arXiv: https://arxiv.org/abs/2311.04913 · HTML: https://mlsanity.com/paper/2311.04913/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 37 references that are in the map.
- First posted on arXiv 1 Nov 2023.

## Builds on (most cited first, 7 of 7)

- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [FinBERT: Financial Sentiment Analysis with Pre-trained Language Models](https://mlsanity.com/paper/1908.10063/) (2019)
- [Understanding AdamW through Proximal Methods and Scale-Freeness](https://mlsanity.com/paper/2202.00089/) (2022)
- [URLNet: Learning a URL Representation with Deep Learning for Malicious URL Detection](https://mlsanity.com/paper/1802.03162/) (2018)
- [URLTran: Improving Phishing URL Detection Using Transformers](https://mlsanity.com/paper/2106.05256/) (2021)
- [Universal Spam Detection using Transfer Learning of BERT Model](https://mlsanity.com/paper/2202.03480/) (2022)

## Built on by (most cited first, 12 of 12)

- [A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2312.02003/) (2023)
- [When LLMs Meet Cybersecurity: A Systematic Literature Review](https://mlsanity.com/paper/2405.03644/) (2024)
- [Large Language Models for Cyber Security: A Systematic Literature Review](https://mlsanity.com/paper/2405.04760/) (2024)
- [An Explainable Transformer-based Model for Phishing Email Detection: A Large Language Model Approach](https://mlsanity.com/paper/2402.13871/) (2024)
- [A Fine-tuning Enhanced RAG System with Quantized Influence Measure as AI Judge](https://mlsanity.com/paper/2402.17081/) (2024)
- [On the Feasibility of Fully AI-automated Vishing Attacks](https://mlsanity.com/paper/2409.13793/) (2024)
- [ExplainableDetector: Exploring Transformer-based Language Modeling Approach for SMS Spam Detection with Explainability Analysis](https://mlsanity.com/paper/2405.08026/) (2024)
- [A Comprehensive Analysis of Adversarial Attacks against Spam Filters](https://mlsanity.com/paper/2505.03831/) (2025)
- [Evaluating LLMs for Police Decision-Making: A Framework Based on Police Action Scenarios](https://mlsanity.com/paper/2601.03553/) (2026)
- [TIPS: Threat Actor Informed Prioritization of Applications using SecEncoder](https://mlsanity.com/paper/2411.07519/) (2024)
- [PLAID: A Unified Data Model for Machine Learning on Heterogeneous Physics Simulations](https://mlsanity.com/paper/2505.02974/) (2025)
- [Trustworthiness Calibration Framework for Phishing Email Detection Using Large Language Models](https://mlsanity.com/paper/2511.04728/) (2025)

## Abstract (arXiv)

Phishing and spam detection is long standing challenge that has been the subject of much academic research. Large Language Models (LLM) have vast potential to transform society and provide new and innovative approaches to solve well-established challenges. Phishing and spam have caused financial hardships and lost time and resources to email users all over the world and frequently serve as an entry point for ransomware threat actors. While detection approaches exist, especially heuristic-based approaches, LLMs offer the potential to venture into a new unexplored area for understanding and solving this challenge. LLMs have rapidly altered the landscape from business, consumers, and throughout academia and demonstrate transformational potential for the potential of society. Based on this, applying these new and innovative approaches to email detection is a rational next step in academic research. In this work, we present IPSDM, our model based on fine-tuning the BERT family of models to specifically detect phishing and spam email. We demonstrate our fine-tuned version, IPSDM, is able to better classify emails in both unbalanced and balanced datasets. This work serves as an important first step towards employing LLMs to improve the security of our information systems.