# Can Features for Phishing URL Detection Be Trusted Across Diverse Datasets? A Case Study with Explainable AI

arXiv: https://arxiv.org/abs/2411.09813 · HTML: https://mlsanity.com/paper/2411.09813/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 41 references that are in the map.
- First posted on arXiv 14 Nov 2024.

## Builds on (most cited first, 6 of 6)

- [From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy](https://mlsanity.com/paper/2307.00691/) (2023)
- [Towards Benchmark Datasets for Machine Learning Based Website Phishing Detection: An experimental study](https://mlsanity.com/paper/2010.12847/) (2020)
- [Look Before You Leap: Detecting Phishing Web Pages by Exploiting Raw URL And HTML Characteristics](https://mlsanity.com/paper/2011.04412/) (2020)
- [AbuseGPT: Abuse of Generative AI ChatBots to Create Smishing Campaigns](https://mlsanity.com/paper/2402.09728/) (2024)
- [Improving DGA-Based Malicious Domain Classifiers for Malware Defense with Adversarial Machine Learning](https://mlsanity.com/paper/2101.00521/) (2021)
- [Data-Driven Characterization and Detection of COVID-19 Themed Malicious Websites](https://mlsanity.com/paper/2102.13226/) (2021)

## Built on by (most cited first, 1 of 1)

- [Explainable but Vulnerable: Adversarial Attacks on XAI Explanation in Cybersecurity Applications](https://mlsanity.com/paper/2510.03623/) (2025)

## Abstract (arXiv)

Phishing has been a prevalent cyber threat that manipulates users into revealing sensitive private information through deceptive tactics, designed to masquerade as trustworthy entities. Over the years, proactively detection of phishing URLs (or websites) has been established as an widely-accepted defense approach. In literature, we often find supervised Machine Learning (ML) models with highly competitive performance for detecting phishing websites based on the extracted features from both phishing and benign (i.e., legitimate) websites. However, it is still unclear if these features or indicators are dependent on a particular dataset or they are generalized for overall phishing detection. In this paper, we delve deeper into this issue by analyzing two publicly available phishing URL datasets, where each dataset has its own set of unique and overlapping features related to URL string and website contents. We want to investigate if overlapping features are similar in nature across datasets and how does the model perform when trained on one dataset and tested on the other. We conduct practical experiments and leverage explainable AI (XAI) methods such as SHAP plots to provide insights into different features' contributions in case of phishing detection to answer our primary question, "Can features for phishing URL detection be trusted across diverse dataset?". Our case study experiment results show that features for phishing URL detection can often be dataset-dependent and thus may not be trusted across different datasets even though they share same set of feature behaviors.