mlsanityData to 7 Oct 2026

SoK: The Impact of Unlabelled Data in Cyberthreat Detection

arXiv 2205.08944 ↗First posted 18 May 2022cs.CR cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Machine learning (ML) has become an important paradigm for cyberthreat detection (CTD) in the recent years. A substantial research effort has been invested in the development of specialized algorithms for CTD tasks. From the operational perspective, however, the progress of ML-based CTD is hindered by the difficulty in obtaining the large sets of labelled data to train ML detectors. A potential solution to this problem are semisupervised learning (SsL) methods, which combine small labelled datasets with large amounts of unlabelled data. This paper is aimed at systematization of existing work on SsL for CTD and, in particular, on understanding the utility of unlabelled data in such systems. To this end, we analyze the cost of labelling in various CTD tasks and develop a formal cost model for SsL in this context. Building on this foundation, we formalize a set of requirements for evaluation of SsL methods, which elucidates the contribution of unlabelled data. We review the state-of-the-art and observe that no previous work meets such requirements. To address this problem, we propose a framework for assessing the benefits of unlabelled data in SsL. We showcase an application of this framework by performing the first benchmark evaluation that highlights the tradeoffs of 9 existing SsL methods on 9 public datasets. Our findings verify that, in some cases, unlabelled data provides a small, but statistically significant, performance gain. This paper highlights that SsL in CTD has a lot of room for improvement, which should stimulate future research in this field.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 26 of its 157 references that are in the map. First posted on arXiv 18 May 2022.

Builds on

26 of 157 references.

References · 26

  1. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
  2. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  3. 2018Detecting and Correcting for Label Shift with Black Box Predictorscited by 364
  4. 2018Audio-Visual Scene Analysis with Self-Supervised Multisensory Featurescited by 335
  5. 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 318
  6. 2020A Metric Learning Reality Checkcited by 317
  7. 2018EMBER: An Open Dataset for Training Static PE Malware Machine Learning Modelscited by 164
  8. 2021The Benchmark Lotterycited by 133
  9. 2018Kitsune: An Ensemble of Autoencoders for Online Network Intrusion Detectioncited by 121
  10. 2018TESSERACT: Eliminating Experimental Bias in Malware Classification across Space and Timecited by 108
  11. 2018Exploring Connections Between Active Learning and Model Extractioncited by 84
  12. 2021Poisoning the Unlabeled Dataset of Semi-Supervised Learningcited by 44
14 more
  1. 2019XGBOD: Improving Supervised Outlier Detection with Unsupervised Representation Learningcited by 42
  2. 2015Reviewer Integration and Performance Measurement for Malware Detectioncited by 19
  3. 2018Active Learning for Wireless IoT Intrusion Detectioncited by 19
  4. 2019SOK: A Comprehensive Reexamination of Phishing Research from the Security Perspectivecited by 15
  5. 2017DeltaPhish: Detecting Phishing Webpages in Compromised Websitescited by 13
  6. 2019Hardening Random Forest Cyber Detectors Against Adversarial Attackscited by 12
  7. 2020Random Partitioning Forest for Point-Wise and Collective Anomaly Detection -- Application to Intrusion Detectioncited by 10
  8. 2016Darknet and Deepnet Mining for Proactive Cybersecurity Threat Intelligencecited by 8
  9. 2019On generating network traffic datasets with synthetic attacks for intrusion detectioncited by 5
  10. 2020Experimental Review of Neural-based approaches for Network Intrusion Managementcited by 5
  11. 2018Optimal Sample Size Planning for the Wilcoxon-Mann-Whitney-Testcited by 3
  12. 2018Deep Learning for Malicious Flow Detectioncited by 2
  13. 2020How Did That Get In My Phone? Unwanted App Distribution on Android Devicescited by 2
  14. 2017Semi-supervised classification for dynamic Android malware detectioncited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 4; 2024: 3; 2025: 3; 2026: 2.

15 papers.

Sort:

  1. 2022"Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practicecited by 55
  2. 2022The Role of Machine Learning in Cybersecuritycited by 21
  3. 2023SoK: Pragmatic Assessment of Machine Learning for Network Intrusion Detectioncited by 13
  4. 2023A Survey on Malware Detection with Graph Representation Learningcited by 11
  5. 2022Multi-SpacePhish: Extending the Evasion-space of Adversarial Attacks against Phishing Website Detectors using Machine Learningcited by 7
  6. 2023Nebula: Self-Attention for Dynamic Malware Analysiscited by 7
  7. 2024SoK: On the Offensive Potential of AIcited by 5
  8. 2026Cybersecurity is the True Frontier for Generative AI Success or Failurecited by 2
  9. 2024AnomalyAID: Reliable Interpretation for Semi-supervised Network Anomaly Detectioncited by 1
  10. 2025ADAPT: A Pseudo-labeling Approach to Combat Concept Drift in Malware Detectioncited by 1
  11. 2023Assessing the Impact of a Supervised Classification Filter on Flow-based Hybrid Network Anomaly Detectioncited by 0
  12. 2024Introducing Adaptive Continuous Adversarial Training (ACAT) to Enhance ML Robustnesscited by 0
3 more
  1. 2025MLRan: A Behavioural Dataset for Ransomware Analysis and Detectioncited by 0
  2. 2025An Investigation into the Performance of Non-Contrastive Self-Supervised Learning Methods for Network Intrusion Detectioncited by 0
  3. 2026Multi-Level Distributional Entropy from Flow Summary Statistics for Explainable Network Intrusion Detectioncited by 0