# Towards a Standard Feature Set for Network Intrusion Detection System Datasets

arXiv: https://arxiv.org/abs/2101.11315 · HTML: https://mlsanity.com/paper/2101.11315/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jan 2021.

## Built on by (most cited first, 30 of 40)

- [Anomal-E: A Self-Supervised Network Intrusion Detection System based on Graph Neural Networks](https://mlsanity.com/paper/2207.06819/) (2022)
- [FlowTransformer: A Transformer Framework for Flow-based Network Intrusion Detection Systems](https://mlsanity.com/paper/2304.14746/) (2023)
- [Adv-Bot: Realistic Adversarial Botnet Attacks against Network Intrusion Detection Systems](https://mlsanity.com/paper/2303.06664/) (2023)
- [Towards Explainable Network Intrusion Detection using Large Language Models](https://mlsanity.com/paper/2408.04342/) (2024)
- [AnoShift: A Distribution Shift Benchmark for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2206.15476/) (2022)
- [Evaluating Standard Feature Sets Towards Increased Generalisability and Explainability of ML-based Network Intrusion Detection](https://mlsanity.com/paper/2104.07183/) (2021)
- [On Generalisability of Machine Learning-based Network Intrusion Detection Systems](https://mlsanity.com/paper/2205.04112/) (2022)
- [Applying Self-supervised Learning to Network Intrusion Detection for Network Flows with Graph Neural Network](https://mlsanity.com/paper/2403.01501/) (2024)
- [eX-NIDS: A Framework for Explainable Network Intrusion Detection Leveraging Large Language Models](https://mlsanity.com/paper/2507.16241/) (2025)
- [On the Cross-Dataset Generalization of Machine Learning for Network Intrusion Detection](https://mlsanity.com/paper/2402.10974/) (2024)
- [Survey of Graph Neural Network for Internet of Things and NextG Networks](https://mlsanity.com/paper/2405.17309/) (2024)
- [Network Intrusion Detection System in a Light Bulb](https://mlsanity.com/paper/2210.03254/) (2022)
- [Investigation and rectification of NIDS datasets and standardized feature set derivation for network attack detection with graph neural networks](https://mlsanity.com/paper/2212.13994/) (2022)
- [Enhancing Trustworthiness in ML-Based Network Intrusion Detection with Uncertainty Quantification](https://mlsanity.com/paper/2310.10655/) (2023)
- [Enhancing IoT Security via Automatic Network Traffic Analysis: The Transition from Machine Learning to Deep Learning](https://mlsanity.com/paper/2312.00034/) (2023)
- [PPT-GNN: A Practical Pre-Trained Spatio-Temporal Graph Neural Network for Network Security](https://mlsanity.com/paper/2406.13365/) (2024)
- [A Critical Assessment of Interpretable and Explainable Machine Learning for Intrusion Detection](https://mlsanity.com/paper/2407.04009/) (2024)
- [Temporal Analysis of NetFlow Datasets for Network Intrusion Detection Systems](https://mlsanity.com/paper/2503.04404/) (2025)
- [Anomaly-Flow: A Multi-domain Federated Generative Adversarial Network for Distributed Denial-of-Service Detection](https://mlsanity.com/paper/2503.14618/) (2025)
- [Self-Supervised Learning of Graph Representations for Network Intrusion Detection](https://mlsanity.com/paper/2509.16625/) (2025)
- [Improving the Reliability of Network Intrusion Detection Systems through Dataset Integration](https://mlsanity.com/paper/2112.02080/) (2021)
- [DOC-NAD: A Hybrid Deep One-class Classifier for Network Anomaly Detection](https://mlsanity.com/paper/2212.07558/) (2022)
- [3D-IDS: Doubly Disentangled Dynamic Intrusion Detection](https://mlsanity.com/paper/2307.11079/) (2023)
- [Integrating Graph Neural Networks with Scattering Transform for Anomaly Detection](https://mlsanity.com/paper/2404.10800/) (2024)
- [MEGA: Message Passing Neural Networks for Multigraphs with EdGe Attributes](https://mlsanity.com/paper/2412.00241/) (2024)
- [Flow Exporter Impact on Intelligent Intrusion Detection Systems](https://mlsanity.com/paper/2412.14021/) (2024)
- [An Evaluation Framework for Network IDS/IPS Datasets: Leveraging MITRE ATT&CK and Industry Relevance Metrics](https://mlsanity.com/paper/2511.12743/) (2025)
- [BS-GAT Behavior Similarity Based Graph Attention Network for Network Intrusion Detection](https://mlsanity.com/paper/2304.07226/) (2023)
- [NIDS Neural Networks Using Sliding Time Window Data Processing with Trainable Activations and its Generalization Capability](https://mlsanity.com/paper/2410.18658/) (2024)
- [Anomaly detection in network flows using unsupervised online machine learning](https://mlsanity.com/paper/2509.01375/) (2025)

## Abstract (arXiv)

Network Intrusion Detection Systems (NIDSs) are important tools for the protection of computer networks against increasingly frequent and sophisticated cyber attacks. Recently, a lot of research effort has been dedicated to the development of Machine Learning (ML) based NIDSs. As in any ML-based application, the availability of high-quality datasets is critical for the training and evaluation of ML-based NIDS. One of the key problems with the currently available datasets is the lack of a standard feature set. The use of a unique and proprietary set of features for each of the publicly available datasets makes it virtually impossible to compare the performance of ML-based traffic classifiers on different datasets, and hence to evaluate the ability of these systems to generalise across different network scenarios. To address that limitation, this paper proposes and evaluates standard NIDS feature sets based on the NetFlow network meta-data collection protocol and system. We evaluate and compare two NetFlow-based feature set variants, a version with 12 features, and another one with 43 features.