# Evaluating Standard Feature Sets Towards Increased Generalisability and Explainability of ML-based Network Intrusion Detection

arXiv: https://arxiv.org/abs/2104.07183 · HTML: https://mlsanity.com/paper/2104.07183/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 34 references that are in the map.
- First posted on arXiv 15 Apr 2021.

## Builds on (most cited first, 7 of 7)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning](https://mlsanity.com/paper/1712.03141/) (2017)
- [Towards the Development of Realistic Botnet Dataset in the Internet of Things for Network Forensic Analytics: Bot-IoT Dataset](https://mlsanity.com/paper/1811.00701/) (2018)
- [NetFlow Datasets for Machine Learning-based Network Intrusion Detection Systems](https://mlsanity.com/paper/2011.09144/) (2020)
- [Towards a Standard Feature Set for Network Intrusion Detection System Datasets](https://mlsanity.com/paper/2101.11315/) (2021)
- [Explaining Network Intrusion Detection System Using Explainable AI Framework](https://mlsanity.com/paper/2103.07110/) (2021)

## Built on by (most cited first, 8 of 8)

- [Applying Self-supervised Learning to Network Intrusion Detection for Network Flows with Graph Neural Network](https://mlsanity.com/paper/2403.01501/) (2024)
- [On the Cross-Dataset Generalization of Machine Learning for Network Intrusion Detection](https://mlsanity.com/paper/2402.10974/) (2024)
- [Temporal Analysis of NetFlow Datasets for Network Intrusion Detection Systems](https://mlsanity.com/paper/2503.04404/) (2025)
- [3D-IDS: Doubly Disentangled Dynamic Intrusion Detection](https://mlsanity.com/paper/2307.11079/) (2023)
- [An Approach To Enhance IoT Security In 6G Networks Through Explainable AI](https://mlsanity.com/paper/2410.05310/) (2024)
- [Exploring Feature Importance and Explainability Towards Enhanced ML-Based DoS Detection in AI Systems](https://mlsanity.com/paper/2411.03355/) (2024)
- [L-XAIDS: A LIME-based eXplainable AI framework for Intrusion Detection Systems](https://mlsanity.com/paper/2508.17244/) (2025)
- [Collaborative Zone-Adaptive Zero-Day Intrusion Detection for IoBT](https://mlsanity.com/paper/2602.16098/) (2026)

## Abstract (arXiv)

Machine Learning (ML)-based network intrusion detection systems bring many benefits for enhancing the cybersecurity posture of an organisation. Many systems have been designed and developed in the research community, often achieving a close to perfect detection rate when evaluated using synthetic datasets. However, the high number of academic research has not often translated into practical deployments. There are several causes contributing towards the wide gap between research and production, such as the limited ability of comprehensive evaluation of ML models and lack of understanding of internal ML operations. This paper tightens the gap by evaluating the generalisability of a common feature set to different network environments and attack scenarios. Therefore, two feature sets (NetFlow and CICFlowMeter) have been evaluated in terms of detection accuracy across three key datasets, i.e., CSE-CIC-IDS2018, BoT-IoT, and ToN-IoT. The results show the superiority of the NetFlow feature set in enhancing the ML models detection accuracy of various network attacks. In addition, due to the complexity of the learning models, SHapley Additive exPlanations (SHAP), an explainable AI methodology, has been adopted to explain and interpret the classification decisions of ML models. The Shapley values of two common feature sets have been analysed across multiple datasets to determine the influence contributed by each feature towards the final ML prediction.