# On Generalisability of Machine Learning-based Network Intrusion Detection Systems

arXiv: https://arxiv.org/abs/2205.04112 · HTML: https://mlsanity.com/paper/2205.04112/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 31 references that are in the map.
- First posted on arXiv 9 May 2022.

## Builds on (most cited first, 11 of 11)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Towards the Development of Realistic Botnet Dataset in the Internet of Things for Network Forensic Analytics: Bot-IoT Dataset](https://mlsanity.com/paper/1811.00701/) (2018)
- [E-GraphSAGE: A Graph Neural Network based Intrusion Detection System for IoT](https://mlsanity.com/paper/2103.16329/) (2021)
- [NetFlow Datasets for Machine Learning-based Network Intrusion Detection Systems](https://mlsanity.com/paper/2011.09144/) (2020)
- [Towards a Standard Feature Set for Network Intrusion Detection System Datasets](https://mlsanity.com/paper/2101.11315/) (2021)
- [From Zero-Shot Machine Learning to Zero-Day Attack Detection](https://mlsanity.com/paper/2109.14868/) (2021)
- [The Cross-evaluation of Machine Learning-based Network Intrusion Detection Systems](https://mlsanity.com/paper/2203.04686/) (2022)
- [A Transfer Learning Approach for Network Intrusion Detection](https://mlsanity.com/paper/1909.02352/) (2019)
- [A Grassmannian Approach to Zero-Shot Learning for Network Intrusion Detection](https://mlsanity.com/paper/1709.07984/) (2017)
- [Exploring Edge TPU for Network Intrusion Detection in IoT](https://mlsanity.com/paper/2103.16295/) (2021)
- [Benchmarking the Benchmark -- Analysis of Synthetic NIDS Datasets](https://mlsanity.com/paper/2104.09029/) (2021)

## Built on by (most cited first, 8 of 8)

- [Anomal-E: A Self-Supervised Network Intrusion Detection System based on Graph Neural Networks](https://mlsanity.com/paper/2207.06819/) (2022)
- [FlowTransformer: A Transformer Framework for Flow-based Network Intrusion Detection Systems](https://mlsanity.com/paper/2304.14746/) (2023)
- [Transformers and Large Language Models for Efficient Intrusion Detection Systems: A Comprehensive Survey](https://mlsanity.com/paper/2408.07583/) (2024)
- [Towards Explainable Network Intrusion Detection using Large Language Models](https://mlsanity.com/paper/2408.04342/) (2024)
- [DI-NIDS: Domain Invariant Network Intrusion Detection System](https://mlsanity.com/paper/2210.08252/) (2022)
- [Generative Active Adaptation for Drifting and Imbalanced Network Intrusion Detection](https://mlsanity.com/paper/2503.03022/) (2025)
- [Integrating Graph Neural Networks with Scattering Transform for Anomaly Detection](https://mlsanity.com/paper/2404.10800/) (2024)
- [Contrastive Self-Supervised Network Intrusion Detection using Augmented Negative Pairs](https://mlsanity.com/paper/2509.06550/) (2025)

## Abstract (arXiv)

Many of the proposed machine learning (ML) based network intrusion detection systems (NIDSs) achieve near perfect detection performance when evaluated on synthetic benchmark datasets. Though, there is no record of if and how these results generalise to other network scenarios, in particular to real-world networks. In this paper, we investigate the generalisability property of ML-based NIDSs by extensively evaluating seven supervised and unsupervised learning models on four recently published benchmark NIDS datasets. Our investigation indicates that none of the considered models is able to generalise over all studied datasets. Interestingly, our results also indicate that the generalisability has a high degree of asymmetry, i.e., swapping the source and target domains can significantly change the classification performance. Our investigation also indicates that overall, unsupervised learning methods generalise better than supervised learning models in our considered scenarios. Using SHAP values to explain these results indicates that the lack of generalisability is mainly due to the presence of strong correspondence between the values of one or more features and Attack/Benign classes in one dataset-model combination and its absence in other datasets that have different feature distributions.