# TrafficGPT: Breaking the Token Barrier for Efficient Long Traffic Analysis and Generation

arXiv: https://arxiv.org/abs/2403.05822 · HTML: https://mlsanity.com/paper/2403.05822/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 72 references that are in the map.
- First posted on arXiv 9 Mar 2024.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [RWKV: Reinventing RNNs for the Transformer Era](https://mlsanity.com/paper/2305.13048/) (2023)
- [Retentive Network: A Successor to Transformer for Large Language Models](https://mlsanity.com/paper/2307.08621/) (2023)
- [Efficient Content-Based Sparse Attention with Routing Transformers](https://mlsanity.com/paper/2003.05997/) (2020)
- [On The Computational Complexity of Self-Attention](https://mlsanity.com/paper/2209.04881/) (2022)
- [Star-Transformer](https://mlsanity.com/paper/1902.09113/) (2019)
- [ET-BERT: A Contextualized Datagram Representation with Pre-training Transformers for Encrypted Traffic Classification](https://mlsanity.com/paper/2202.06335/) (2022)
- [Poolingformer: Long Document Modeling with Pooling Attention](https://mlsanity.com/paper/2105.04371/) (2021)
- [k-fingerprinting: a Robust Scalable Website Fingerprinting Technique](https://mlsanity.com/paper/1509.00789/) (2015)
- [NetGPT: Generative Pretrained Transformer for Network Traffic](https://mlsanity.com/paper/2304.09513/) (2023)
- [Flow-based Network Traffic Generation using Generative Adversarial Networks](https://mlsanity.com/paper/1810.07795/) (2018)
- [New Directions in Automated Traffic Analysis](https://mlsanity.com/paper/2008.02695/) (2020)
- [Simple Local Attentions Remain Competitive for Long-Context Tasks](https://mlsanity.com/paper/2112.07210/) (2021)
- [Automated Website Fingerprinting through Deep Learning](https://mlsanity.com/paper/1708.06376/) (2017)
- [netFound: Principled Design for Network Foundation Models](https://mlsanity.com/paper/2310.17025/) (2023)
- [Lens: A Knowledge-Guided Foundation Model for Network Traffic](https://mlsanity.com/paper/2402.03646/) (2024)
- [Your Smart Home Can't Keep a Secret: Towards Automated Fingerprinting of IoT Traffic with Neural Networks](https://mlsanity.com/paper/1909.00104/) (2019)
- [Generating Practical Adversarial Network Traffic Flows Using NIDSGAN](https://mlsanity.com/paper/2203.06694/) (2022)
- [ZLeaks: Passive Inference Attacks on Zigbee based Smart Homes](https://mlsanity.com/paper/2107.10830/) (2021)
- [Fingerprinting Search Keywords over HTTPS at Scale](https://mlsanity.com/paper/2008.08161/) (2020)

## Built on by (most cited first, 15 of 15)

- [NetMamba: Efficient Network Traffic Classification via Pre-training Unidirectional Mamba](https://mlsanity.com/paper/2405.11449/) (2024)
- [Mapping the Landscape of Generative AI in Network Monitoring and Management](https://mlsanity.com/paper/2502.08576/) (2025)
- [Large Language Models for Network Intrusion Detection Systems: Foundations, Implementations, and Future Directions](https://mlsanity.com/paper/2507.04752/) (2025)
- [Quantifying the Privacy Implications of High-Fidelity Synthetic Network Traffic](https://mlsanity.com/paper/2511.20497/) (2025)
- [A Comprehensive Survey on Network Traffic Synthesis: From Statistical Models to Deep Learning](https://mlsanity.com/paper/2507.01976/) (2025)
- [Demystifying Network Foundation Models](https://mlsanity.com/paper/2509.23089/) (2025)
- [PLUME: Building a Network-Native Foundation Model for Wireless Traces via Protocol-Aware Tokenization](https://mlsanity.com/paper/2603.13647/) (2026)
- [Application of Tabular Transformer Architectures for Operating System Fingerprinting](https://mlsanity.com/paper/2502.09084/) (2025)
- [NetMamba+: A Framework of Pre-trained Models for Efficient and Accurate Network Traffic Classification](https://mlsanity.com/paper/2601.21792/) (2026)
- [Harnessing Generative Pre-Trained Transformer for Datacenter Packet Trace Generation](https://mlsanity.com/paper/2501.12033/) (2025)
- [Versatile yet Efficient Network Traffic Analysis: Offloading Network Foundation Model to SmartNIC](https://mlsanity.com/paper/2508.02001/) (2025)
- [MalRAG: A Retrieval-Augmented LLM Framework for Open-set Malicious Traffic Identification](https://mlsanity.com/paper/2511.14129/) (2025)
- [Lightweight GenAI for Network Traffic Generation: Fidelity, Augmentation, and Classification](https://mlsanity.com/paper/2603.25507/) (2026)
- [FedLLM: A Privacy-Preserving Federated Large Language Model for Explainable Traffic Flow Prediction](https://mlsanity.com/paper/2604.16612/) (2026)
- [Protocol-Aware Tokenization and Architecture Co-Design for Wireless Packet Foundation Models](https://mlsanity.com/paper/2606.20587/) (2026)

## Abstract (arXiv)

Over the years, network traffic analysis and generation have advanced significantly. From traditional statistical methods, the field has progressed to sophisticated deep learning techniques. This progress has improved the ability to detect complex patterns and security threats, as well as to test and optimize network performance. However, obstacles persist, such as the dependence on labeled data for analysis and the difficulty of generating traffic samples that follow realistic patterns. Pre-trained deep neural networks have emerged as powerful tools to resolve these issues, offering improved performance by learning robust data representations from large unlabeled datasets. Despite their benefits, existing pre-trained models face challenges like token length limitation, which restricts their usefulness in comprehensive traffic analysis and realistic traffic generation. To address these challenges, we introduce TrafficGPT, a deep learning model that can tackle complex challenges related to long flow classification and generation tasks. This model uses generative pre-training with the linear attention mechanism, which allows for a substantially increased capacity of up to 12,032 tokens from the previous limit of only 512 tokens. TrafficGPT demonstrates superior performance in classification tasks, reaching state-of-the-art levels. In generation tasks, it closely resembles real traffic flows, with low JS divergence and an F1 score close to 0.5 (representing a random guess) in discriminating generated data. These advancements hold promise for future applications in both traffic flow classification and generation tasks.