# Packet Inspection Transformer: A Self-Supervised Journey to Unseen Malware Detection with Few Samples

arXiv: https://arxiv.org/abs/2409.18219 · HTML: https://mlsanity.com/paper/2409.18219/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 55 references that are in the map.
- First posted on arXiv 26 Sep 2024.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends](https://mlsanity.com/paper/2301.05712/) (2023)
- [Laplacian Regularized Few-Shot Learning](https://mlsanity.com/paper/2006.15486/) (2020)
- [Deep Packet: A Novel Approach For Encrypted Traffic Classification Using Deep Learning](https://mlsanity.com/paper/1709.02656/) (2017)
- [On Episodes, Prototypical Networks, and Few-shot Learning](https://mlsanity.com/paper/2012.09831/) (2020)
- [Billion-Scale Pretraining with Vision Transformers for Multi-Task Visual Representations](https://mlsanity.com/paper/2108.05887/) (2021)
- [Machine Learning DDoS Detection for Consumer Internet of Things Devices](https://mlsanity.com/paper/1804.04159/) (2018)
- [Towards Novel Malicious Packet Recognition: A Few-Shot Learning Approach](https://mlsanity.com/paper/2409.11254/) (2024)

## Built on by (most cited first, 2 of 2)

- [Adapter-Based Few-Shot Continual Learning for Malicious Packet Recognition](https://mlsanity.com/paper/2608.23536/) (2026)
- [Temporal and Multimodal Deep Learning for Cyberattack Detection in LEO Satellite Systems](https://mlsanity.com/paper/2609.10746/) (2026)

## Abstract (arXiv)

As networks continue to expand and become more interconnected, the need for novel malware detection methods becomes more pronounced. Traditional security measures are increasingly inadequate against the sophistication of modern cyber attacks. Deep Packet Inspection (DPI) has been pivotal in enhancing network security, offering an in-depth analysis of network traffic that surpasses conventional monitoring techniques. DPI not only examines the metadata of network packets, but also dives into the actual content being carried within the packet payloads, providing a comprehensive view of the data flowing through networks. While the integration of advanced deep learning techniques with DPI has introduced modern methodologies into malware detection and network traffic classification, state-of-the-art supervised learning approaches are limited by their reliance on large amounts of annotated data and their inability to generalize to novel, unseen malware threats. To address these limitations, this paper leverages the recent advancements in self-supervised learning (SSL) and few-shot learning (FSL). Our proposed self-supervised approach trains a transformer via SSL to learn the embedding of packet content, including payload, from vast amounts of unlabeled data by masking portions of packets, leading to a learned representation that generalizes to various downstream tasks. Once the representation is extracted from the packets, they are used to train a malware detection algorithm. The representation obtained from the transformer is then used to adapt the malware detector to novel types of attacks using few-shot learning approaches. Our experimental results demonstrate that our method achieves classification accuracies of up to 94.76% on the UNSW-NB15 dataset and 83.25% on the CIC-IoT23 dataset.