# A Transformer-Based Framework for Payload Malware Detection and Classification

arXiv: https://arxiv.org/abs/2403.18223 · HTML: https://mlsanity.com/paper/2403.18223/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 22 references that are in the map.
- First posted on arXiv 27 Mar 2024.

## Builds on (most cited first, 4 of 4)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Deep Packet: A Novel Approach For Encrypted Traffic Classification Using Deep Learning](https://mlsanity.com/paper/1709.02656/) (2017)
- [Machine Learning DDoS Detection for Consumer Internet of Things Devices](https://mlsanity.com/paper/1804.04159/) (2018)

## Built on by (most cited first, 4 of 4)

- [Fine-tuned Large Language Models (LLMs): Improved Prompt Injection Attacks Detection](https://mlsanity.com/paper/2410.21337/) (2024)
- [Towards Novel Malicious Packet Recognition: A Few-Shot Learning Approach](https://mlsanity.com/paper/2409.11254/) (2024)
- [Testbed and Software Architecture for Enhancing Security in Industrial Private 5G Networks](https://mlsanity.com/paper/2507.20873/) (2025)
- [A Structured State Space Sequence Model for Multi-Class Classification of Malware](https://mlsanity.com/paper/2610.01893/) (2026)

## Abstract (arXiv)

As malicious cyber threats become more sophisticated in breaching computer networks, the need for effective intrusion detection systems (IDSs) becomes crucial. Techniques such as Deep Packet Inspection (DPI) have been introduced to allow IDSs analyze the content of network packets, providing more context for identifying potential threats. IDSs traditionally rely on using anomaly-based and signature-based detection techniques to detect unrecognized and suspicious activity. Deep learning techniques have shown great potential in DPI for IDSs due to their efficiency in learning intricate patterns from the packet content being transmitted through the network. In this paper, we propose a revolutionary DPI algorithm based on transformers adapted for the purpose of detecting malicious traffic with a classifier head. Transformers learn the complex content of sequence data and generalize them well to similar scenarios thanks to their self-attention mechanism. Our proposed method uses the raw payload bytes that represent the packet contents and is deployed as man-in-the-middle. The payload bytes are used to detect malicious packets and classify their types. Experimental results on the UNSW-NB15 and CIC-IOT23 datasets demonstrate that our transformer-based model is effective in distinguishing malicious from benign traffic in the test dataset, attaining an average accuracy of 79% using binary classification and 72% on the multi-classification experiment, both using solely payload bytes.