# FlowTransformer: A Transformer Framework for Flow-based Network Intrusion Detection Systems

arXiv: https://arxiv.org/abs/2304.14746 · HTML: https://mlsanity.com/paper/2304.14746/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 45 references that are in the map.
- First posted on arXiv 28 Apr 2023.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [Graph Transformer Networks](https://mlsanity.com/paper/1911.06455/) (2019)
- [TabTransformer: Tabular Data Modeling Using Contextual Embeddings](https://mlsanity.com/paper/2012.06678/) (2020)
- [ML-Decoder: Scalable and Versatile Classification Head](https://mlsanity.com/paper/2111.12933/) (2021)
- [A survey of dimensionality reduction techniques](https://mlsanity.com/paper/1403.2877/) (2014)
- [Towards a Standard Feature Set for Network Intrusion Detection System Datasets](https://mlsanity.com/paper/2101.11315/) (2021)
- [On Generalisability of Machine Learning-based Network Intrusion Detection Systems](https://mlsanity.com/paper/2205.04112/) (2022)
- [DI-NIDS: Domain Invariant Network Intrusion Detection System](https://mlsanity.com/paper/2210.08252/) (2022)
- [Group Generalized Mean Pooling for Vision Transformer](https://mlsanity.com/paper/2212.04114/) (2022)
- [Network Intrusion Detection System in a Light Bulb](https://mlsanity.com/paper/2210.03254/) (2022)

## Built on by (most cited first, 24 of 24)

- [Large Language Models for Cyber Security: A Systematic Literature Review](https://mlsanity.com/paper/2405.04760/) (2024)
- [Transformers and Large Language Models for Efficient Intrusion Detection Systems: A Comprehensive Survey](https://mlsanity.com/paper/2408.07583/) (2024)
- [Moderating New Waves of Online Hate with Chain-of-Thought Reasoning in Large Language Models](https://mlsanity.com/paper/2312.15099/) (2023)
- [Towards Explainable Network Intrusion Detection using Large Language Models](https://mlsanity.com/paper/2408.04342/) (2024)
- [eX-NIDS: A Framework for Explainable Network Intrusion Detection Leveraging Large Language Models](https://mlsanity.com/paper/2507.16241/) (2025)
- [Mapping the Landscape of Generative AI in Network Monitoring and Management](https://mlsanity.com/paper/2502.08576/) (2025)
- [Towards Practical Multi-label Causal Discovery in High-Dimensional Event Sequences via One-Shot Graph Aggregation](https://mlsanity.com/paper/2509.19112/) (2025)
- [One-Shot Multi-Label Causal Discovery in High-Dimensional Event Sequences](https://mlsanity.com/paper/2509.23213/) (2025)
- [Temporal Analysis of NetFlow Datasets for Network Intrusion Detection Systems](https://mlsanity.com/paper/2503.04404/) (2025)
- [Self-Supervised Transformer-based Contrastive Learning for Intrusion Detection Systems](https://mlsanity.com/paper/2505.08816/) (2025)
- [Interpretable Anomaly-Based DDoS Detection in AI-RAN with XAI and LLMs](https://mlsanity.com/paper/2507.21193/) (2025)
- [ThreatFormer-IDS: Robust Transformer Intrusion Detection with Zero-Day Generalization and Explainable Attribution](https://mlsanity.com/paper/2603.00185/) (2026)
- [Optimized detection of cyber-attacks on IoT networks via hybrid deep learning models](https://mlsanity.com/paper/2502.11470/) (2025)
- [Dynamic Temporal Positional Encodings for Early Intrusion Detection in IoT](https://mlsanity.com/paper/2506.18114/) (2025)
- [A-THENA: Early Intrusion Detection for IoT with Time-Aware Hybrid Encoding and Network-Specific Augmentation](https://mlsanity.com/paper/2604.21623/) (2026)
- [MAWIFlow Benchmark: Realistic Flow-Based Evaluation for Network Intrusion Detection](https://mlsanity.com/paper/2506.17041/) (2025)
- [Hybrid Quantum-Classical Autoencoders for Unsupervised Network Intrusion Detection](https://mlsanity.com/paper/2512.05069/) (2025)
- [Transforming Vehicle Diagnostics: A Multimodal Approach to Error Patterns Prediction](https://mlsanity.com/paper/2602.01109/) (2026)
- [Improving DNS Exfiltration Detection via Transformer Pretraining](https://mlsanity.com/paper/2604.09849/) (2026)
- [MambaNetBurst: Direct Byte-level Network Traffic Classification without Tokenization or Pretraining](https://mlsanity.com/paper/2605.11034/) (2026)
- [Contrastive Learning and Correlation Clustering for Sequences of Network Telescope Data](https://mlsanity.com/paper/2606.04733/) (2026)
- [GenTI: Benchmarking LLMs for Autonomous IDPS Rule Generation for Unseen Attacks](https://mlsanity.com/paper/2606.05844/) (2026)
- [Generative AI and Federated Learning for Intrusion Detection Systems: A Survey](https://mlsanity.com/paper/2607.01305/) (2026)
- [Defending Retrieval-Augmented Intrusion Detection Against Knowledge Poisoning and Prompt Injection](https://mlsanity.com/paper/2608.08100/) (2026)

## Abstract (arXiv)

This paper presents the FlowTransformer framework, a novel approach for implementing transformer-based Network Intrusion Detection Systems (NIDSs). FlowTransformer leverages the strengths of transformer models in identifying the long-term behaviour and characteristics of networks, which are often overlooked by most existing NIDSs. By capturing these complex patterns in network traffic, FlowTransformer offers a flexible and efficient tool for researchers and practitioners in the cybersecurity community who are seeking to implement NIDSs using transformer-based models. FlowTransformer allows the direct substitution of various transformer components, including the input encoding, transformer, classification head, and the evaluation of these across any flow-based network dataset. To demonstrate the effectiveness and efficiency of the FlowTransformer framework, we utilise it to provide an extensive evaluation of various common transformer architectures, such as GPT 2.0 and BERT, on three commonly used public NIDS benchmark datasets. We provide results for accuracy, model size and speed. A key finding of our evaluation is that the choice of classification head has the most significant impact on the model performance. Surprisingly, Global Average Pooling, which is commonly used in text classification, performs very poorly in the context of NIDS. In addition, we show that model size can be reduced by over 50%, and inference and training times improved, with no loss of accuracy, by making specific choices of input encoding and classification head instead of other commonly used alternatives.