# NetGPT: Generative Pretrained Transformer for Network Traffic

arXiv: https://arxiv.org/abs/2304.09513 · HTML: https://mlsanity.com/paper/2304.09513/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 25 references that are in the map.
- First posted on arXiv 19 Apr 2023.

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [ET-BERT: A Contextualized Datagram Representation with Pre-training Transformers for Encrypted Traffic Classification](https://mlsanity.com/paper/2202.06335/) (2022)
- [Flow-based Network Traffic Generation using Generative Adversarial Networks](https://mlsanity.com/paper/1810.07795/) (2018)
- [Realtime Robust Malicious Traffic Detection via Frequency Domain Analysis](https://mlsanity.com/paper/2106.14707/) (2021)
- [Autonomous Unknown-Application Filtering and Labeling for DL-based Traffic Classifier Update](https://mlsanity.com/paper/2002.06359/) (2020)
- [STAN: Synthetic Network Traffic Generation with Generative Neural Models](https://mlsanity.com/paper/2009.12740/) (2020)
- [HEDGE: Efficient Traffic Classification of Encrypted and Compressed Packets](https://mlsanity.com/paper/1905.11873/) (2019)
- [Synthetic flow-based cryptomining attack generation through Generative Adversarial Networks](https://mlsanity.com/paper/2107.14776/) (2021)

## Built on by (most cited first, 30 of 31)

- [NetMamba: Efficient Network Traffic Classification via Pre-training Unidirectional Mamba](https://mlsanity.com/paper/2405.11449/) (2024)
- [FLM-101B: An Open LLM and How to Train It with 100K Budget](https://mlsanity.com/paper/2309.03852/) (2023)
- [TrafficGPT: Breaking the Token Barrier for Efficient Long Traffic Analysis and Generation](https://mlsanity.com/paper/2403.05822/) (2024)
- [Lens: A Knowledge-Guided Foundation Model for Network Traffic](https://mlsanity.com/paper/2402.03646/) (2024)
- [Mapping the Landscape of Generative AI in Network Monitoring and Management](https://mlsanity.com/paper/2502.08576/) (2025)
- [Large Language Models for Network Intrusion Detection Systems: Foundations, Implementations, and Future Directions](https://mlsanity.com/paper/2507.04752/) (2025)
- [Quantifying the Privacy Implications of High-Fidelity Synthetic Network Traffic](https://mlsanity.com/paper/2511.20497/) (2025)
- [Network Anomaly Traffic Detection via Multi-view Feature Fusion](https://mlsanity.com/paper/2409.08020/) (2024)
- [TeleOracle: Fine-Tuned Retrieval-Augmented Generation with Long-Context Support for Network](https://mlsanity.com/paper/2411.02617/) (2024)
- [NetFlowGen: Leveraging Generative Pre-training for Network Traffic Dynamics](https://mlsanity.com/paper/2412.20635/) (2024)
- [Self-Supervised Transformer-based Contrastive Learning for Intrusion Detection Systems](https://mlsanity.com/paper/2505.08816/) (2025)
- [A Comprehensive Survey on Network Traffic Synthesis: From Statistical Models to Deep Learning](https://mlsanity.com/paper/2507.01976/) (2025)
- [Demystifying Network Foundation Models](https://mlsanity.com/paper/2509.23089/) (2025)
- [PLUME: Building a Network-Native Foundation Model for Wireless Traces via Protocol-Aware Tokenization](https://mlsanity.com/paper/2603.13647/) (2026)
- [CENTILE: A Telemetry Foundation Model Evaluated by the Decisions It Drives](https://mlsanity.com/paper/2608.01725/) (2026)
- [MERLOT: A Distilled LLM-based Mixture-of-Experts Framework for Scalable Encrypted Traffic Classification](https://mlsanity.com/paper/2411.13004/) (2024)
- [Application of Tabular Transformer Architectures for Operating System Fingerprinting](https://mlsanity.com/paper/2502.09084/) (2025)
- [UniNet: A Unified Multi-granular Traffic Modeling Framework for Network Security](https://mlsanity.com/paper/2503.04174/) (2025)
- [NetMamba+: A Framework of Pre-trained Models for Efficient and Accurate Network Traffic Classification](https://mlsanity.com/paper/2601.21792/) (2026)
- [TrafficMoE: Heterogeneity-aware Mixture of Experts for Encrypted Traffic Classification](https://mlsanity.com/paper/2603.29520/) (2026)
- [UniAlign: A Model-Agnostic Framework for Robust Network Traffic Classification under Distribution Shifts](https://mlsanity.com/paper/2605.17575/) (2026)
- [SwiftQueue: Optimizing Low-Latency Applications with Swift Packet Queuing](https://mlsanity.com/paper/2410.06112/) (2024)
- [Harnessing Generative Pre-Trained Transformer for Datacenter Packet Trace Generation](https://mlsanity.com/paper/2501.12033/) (2025)
- [Versatile yet Efficient Network Traffic Analysis: Offloading Network Foundation Model to SmartNIC](https://mlsanity.com/paper/2508.02001/) (2025)
- [MalRAG: A Retrieval-Augmented LLM Framework for Open-set Malicious Traffic Identification](https://mlsanity.com/paper/2511.14129/) (2025)
- [Improving DNS Exfiltration Detection via Transformer Pretraining](https://mlsanity.com/paper/2604.09849/) (2026)
- [MambaNetBurst: Direct Byte-level Network Traffic Classification without Tokenization or Pretraining](https://mlsanity.com/paper/2605.11034/) (2026)
- [Contrastive Learning and Correlation Clustering for Sequences of Network Telescope Data](https://mlsanity.com/paper/2606.04733/) (2026)
- [Protocol-Aware Tokenization and Architecture Co-Design for Wireless Packet Foundation Models](https://mlsanity.com/paper/2606.20587/) (2026)
- [PCAP-LM: An LLM-Native Text Representation for TLS Bulk Traffic Analysis](https://mlsanity.com/paper/2607.28100/) (2026)

## Abstract (arXiv)

All data on the Internet are transferred by network traffic, thus accurately modeling network traffic can help improve network services quality and protect data privacy. Pretrained models for network traffic can utilize large-scale raw data to learn the essential characteristics of network traffic, and generate distinguishable results for input traffic without considering specific downstream tasks. Effective pretrained models can significantly optimize the training efficiency and effectiveness of downstream tasks, such as application classification, attack detection and traffic generation. Despite the great success of pretraining in natural language processing, there is no work in the network field. Considering the diverse demands and characteristics of network traffic and network tasks, it is non-trivial to build a pretrained model for network traffic and we face various challenges, especially the heterogeneous headers and payloads in the multi-pattern network traffic and the different dependencies for contexts of diverse downstream network tasks. To tackle these challenges, in this paper, we make the first attempt to provide a generative pretrained model NetGPT for both traffic understanding and generation tasks. We propose the multi-pattern network traffic modeling to construct unified text inputs and support both traffic understanding and generation tasks. We further optimize the adaptation effect of the pretrained model to diversified tasks by shuffling header fields, segmenting packets in flows, and incorporating diverse task labels with prompts. With diverse traffic datasets from encrypted software, DNS, private industrial protocols and cryptocurrency mining, expensive experiments demonstrate the effectiveness of our NetGPT in a range of traffic understanding and generation tasks on traffic datasets, and outperform state-of-the-art baselines by a wide margin.