# Towards a graph-based foundation model for network traffic analysis

arXiv: https://arxiv.org/abs/2409.08111 · HTML: https://mlsanity.com/paper/2409.08111/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 24 references that are in the map.
- First posted on arXiv 12 Sep 2024.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [Towards the Development of Realistic Botnet Dataset in the Internet of Things for Network Forensic Analytics: Bot-IoT Dataset](https://mlsanity.com/paper/1811.00701/) (2018)
- [ET-BERT: A Contextualized Datagram Representation with Pre-training Transformers for Encrypted Traffic Classification](https://mlsanity.com/paper/2202.06335/) (2022)
- [E-GraphSAGE: A Graph Neural Network based Intrusion Detection System for IoT](https://mlsanity.com/paper/2103.16329/) (2021)
- [Anomal-E: A Self-Supervised Network Intrusion Detection System based on Graph Neural Networks](https://mlsanity.com/paper/2207.06819/) (2022)
- [netFound: Principled Design for Network Foundation Models](https://mlsanity.com/paper/2310.17025/) (2023)
- [Always be Pre-Training: Representation Learning for Network Intrusion Detection with GNNs](https://mlsanity.com/paper/2402.18986/) (2024)
- [PPT-GNN: A Practical Pre-Trained Spatio-Temporal Graph Neural Network for Network Security](https://mlsanity.com/paper/2406.13365/) (2024)

## Built on by (most cited first, 2 of 2)

- [Adaptive Initial Residual Connections for GNNs with Theoretical Guarantees](https://mlsanity.com/paper/2511.06598/) (2025)
- [Application of Tabular Transformer Architectures for Operating System Fingerprinting](https://mlsanity.com/paper/2502.09084/) (2025)

## Abstract (arXiv)

Foundation models have shown great promise in various fields of study. A potential application of such models is in computer network traffic analysis, where these models can grasp the complexities of network traffic dynamics and adapt to any specific task or network environment with minimal fine-tuning. Previous approaches have used tokenized hex-level packet data and the model architecture of large language transformer models. We propose a new, efficient graph-based alternative at the flow-level. Our approach represents network traffic as a dynamic spatio-temporal graph, employing a self-supervised link prediction pretraining task to capture the spatial and temporal dynamics in this network graph framework. To evaluate the effectiveness of our approach, we conduct a few-shot learning experiment for three distinct downstream network tasks: intrusion detection, traffic classification, and botnet classification. Models finetuned from our pretrained base achieve an average performance increase of 6.87% over training from scratch, demonstrating their ability to effectively learn general network traffic dynamics during pretraining. This success suggests the potential for a large-scale version to serve as an operational foundational model.