# PhreshPhish: A Real-World, High-Quality, Large-Scale Phishing Website Dataset and Benchmark

arXiv: https://arxiv.org/abs/2507.10854 · HTML: https://mlsanity.com/paper/2507.10854/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 54 references that are in the map.
- First posted on arXiv 14 Jul 2025.

## Builds on (most cited first, 9 of 9)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale](https://mlsanity.com/paper/2406.17557/) (2024)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [Leakage and the Reproducibility Crisis in ML-based Science](https://mlsanity.com/paper/2207.07048/) (2022)
- [KnowPhish: Large Language Models Meet Multimodal Knowledge Graphs for Enhancing Reference-Based Phishing Detection](https://mlsanity.com/paper/2403.02253/) (2024)
- [PhishAgent: A Robust Multimodal Agent for Phishing Webpage Detection](https://mlsanity.com/paper/2408.10738/) (2024)
- [SOK: A Comprehensive Reexamination of Phishing Research from the Security Perspective](https://mlsanity.com/paper/1911.00953/) (2019)
- [Data Commons](https://mlsanity.com/paper/2309.13054/) (2023)
- [PhishIntel: Toward Practical Deployment of Reference-Based Phishing Detection](https://mlsanity.com/paper/2412.09057/) (2024)

## Built on by (most cited first, 2 of 2)

- [CIC-Trap4Phish: A Unified Multi-Format Dataset for Phishing and Quishing Attachment Detection](https://mlsanity.com/paper/2602.09015/) (2026)
- [AURA: Adaptive Uncertainty-Routed Analysis for Email Threat Detection](https://mlsanity.com/paper/2609.19873/) (2026)

## Abstract (arXiv)

Phishing remains a pervasive and growing threat, inflicting heavy economic and reputational damage. While machine learning has been effective in real-time detection of phishing attacks, progress is hindered by lack of large, high-quality datasets and benchmarks. In addition to poor-quality due to challenges in data collection, existing datasets suffer from leakage and unrealistic base rates, leading to overly optimistic performance results. In this paper, we introduce PhreshPhish, a large-scale, high-quality dataset of phishing websites that addresses these limitations. Compared to existing public datasets, PhreshPhish is substantially larger and provides significantly higher quality, as measured by the estimated rate of invalid or mislabeled data points. Additionally, we propose a comprehensive suite of benchmark datasets specifically designed for realistic model evaluation by minimizing leakage, increasing task difficulty, enhancing dataset diversity, and adjustment of base rates more likely to be seen in the real world. We train and evaluate multiple solution approaches to provide baseline performance on the benchmark sets. We believe the availability of this dataset and benchmarks will enable realistic, standardized model comparison and foster further advances in phishing detection. The datasets and benchmarks are available on Hugging Face (https://huggingface.co/datasets/phreshphish/phreshphish).