# DomURLs_BERT: Pre-trained BERT-based Model for Malicious Domains and URLs Detection and Classification

arXiv: https://arxiv.org/abs/2409.09143 · HTML: https://mlsanity.com/paper/2409.09143/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 63 references that are in the map.
- First posted on arXiv 13 Sep 2024.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https://mlsanity.com/paper/2306.01116/) (2023)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [Large Language Models: A Survey](https://mlsanity.com/paper/2402.06196/) (2024)
- [Large Language Models for Cyber Security: A Systematic Literature Review](https://mlsanity.com/paper/2405.04760/) (2024)
- [Tranco: A Research-Oriented Top Sites Ranking Hardened Against Manipulation](https://mlsanity.com/paper/1806.01156/) (2018)
- [CharBERT: Character-aware Pre-trained Language Model](https://mlsanity.com/paper/2011.01513/) (2020)
- [SecureBERT: A Domain-Specific Language Model for Cybersecurity](https://mlsanity.com/paper/2204.02685/) (2022)
- [URLNet: Learning a URL Representation with Deep Learning for Malicious URL Detection](https://mlsanity.com/paper/1802.03162/) (2018)
- [CySecBERT: A Domain-Adapted Language Model for the Cybersecurity Domain](https://mlsanity.com/paper/2212.02974/) (2022)
- [The Role of Machine Learning in Cybersecurity](https://mlsanity.com/paper/2206.09707/) (2022)
- [URLTran: Improving Phishing URL Detection Using Transformers](https://mlsanity.com/paper/2106.05256/) (2021)
- [Malicious URL Detection using Machine Learning: A Survey](https://mlsanity.com/paper/1701.07179/) (2017)
- [DeepDGA: Adversarially-Tuned Domain Generation and Detection](https://mlsanity.com/paper/1610.01969/) (2016)
- [MalBERT: Using Transformers for Cybersecurity and Malicious Software Detection](https://mlsanity.com/paper/2103.03806/) (2021)
- [CharBot: A Simple and Effective Method for Evading DGA Classifiers](https://mlsanity.com/paper/1905.01078/) (2019)
- [Mitigating Adversarial Gray-Box Attacks Against Phishing Detectors](https://mlsanity.com/paper/2212.05380/) (2022)
- [MaskDGA: A Black-box Evasion Technique Against DGA Classifiers and Adversarial Defenses](https://mlsanity.com/paper/1902.08909/) (2019)

## Built on by (most cited first, 5 of 5)

- [From Past to Present: A Survey of Malicious URL Detection Techniques, Datasets and Code Repositories](https://mlsanity.com/paper/2504.16449/) (2025)
- [The Application of Transformer-Based Models for Predicting Consequences of Cyber Attacks](https://mlsanity.com/paper/2508.13030/) (2025)
- [Benchmarking Large Language Models for Zero-shot and Few-shot Phishing URL Detection](https://mlsanity.com/paper/2602.02641/) (2026)
- [Improving DNS Exfiltration Detection via Transformer Pretraining](https://mlsanity.com/paper/2604.09849/) (2026)
- [Domain-specific Pretraining Profile and Transformer Performance: Evidence from Modeling Digital Pragmatics in Arabic-English Code-switching](https://mlsanity.com/paper/2609.14571/) (2026)

## Abstract (arXiv)

Detecting and classifying suspicious or malicious domain names and URLs is fundamental task in cybersecurity. To leverage such indicators of compromise, cybersecurity vendors and practitioners often maintain and update blacklists of known malicious domains and URLs. However, blacklists frequently fail to identify emerging and obfuscated threats. Over the past few decades, there has been significant interest in developing machine learning models that automatically detect malicious domains and URLs, addressing the limitations of blacklists maintenance and updates. In this paper, we introduce DomURLs_BERT, a pre-trained BERT-based encoder adapted for detecting and classifying suspicious/malicious domains and URLs. DomURLs_BERT is pre-trained using the Masked Language Modeling (MLM) objective on a large multilingual corpus of URLs, domain names, and Domain Generation Algorithms (DGA) dataset. In order to assess the performance of DomURLs_BERT, we have conducted experiments on several binary and multi-class classification tasks involving domain names and URLs, covering phishing, malware, DGA, and DNS tunneling. The evaluations results show that the proposed encoder outperforms state-of-the-art character-based deep learning models and cybersecurity-focused BERT models across multiple tasks and datasets. The pre-training dataset, the pre-trained DomURLs_BERT encoder, and the experiments source code are publicly available.