# On the Effect of Ruleset Tuning and Data Imbalance on Explainable Network Security Alert Classifications: a Case-Study on DeepCASE

arXiv: https://arxiv.org/abs/2507.01571 · HTML: https://mlsanity.com/paper/2507.01571/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 42 references that are in the map.
- First posted on arXiv 2 Jul 2025.

## Builds on (most cited first, 5 of 5)

- [TESSERACT: Eliminating Experimental Bias in Malware Classification across Space and Time](https://mlsanity.com/paper/1807.07838/) (2018)
- [How Complex is your classification problem? A survey on measuring classification complexity](https://mlsanity.com/paper/1808.03591/) (2018)
- [Transcending Transcend: Revisiting Malware Classification in the Presence of Concept Drift](https://mlsanity.com/paper/2010.03856/) (2020)
- [SoK: Explainable Machine Learning for Computer Security Applications](https://mlsanity.com/paper/2208.10605/) (2022)
- [Ruling the Unruly: Designing Effective, Low-Noise Network Intrusion Detection Rules for Security Operations Centers](https://mlsanity.com/paper/2501.09808/) (2025)

## Abstract (arXiv)

Automation in Security Operations Centers (SOCs) plays a prominent role in alert classification and incident escalation. However, automated methods must be robust in the presence of imbalanced input data, which can negatively affect performance. Additionally, automated methods should make explainable decisions. In this work, we evaluate the effect of label imbalance on the classification of network intrusion alerts. As our use-case we employ DeepCASE, the state-of-the-art method for automated alert classification. We show that label imbalance impacts both classification performance and correctness of the classification explanations offered by DeepCASE. We conclude tuning the detection rules used in SOCs can significantly reduce imbalance and may benefit the performance and explainability offered by alert post-processing methods such as DeepCASE. Therefore, our findings suggest that traditional methods to improve the quality of input data can benefit automation.