# Implementing Active Learning in Cybersecurity: Detecting Anomalies in Redacted Emails

arXiv: https://arxiv.org/abs/2303.00870 · HTML: https://mlsanity.com/paper/2303.00870/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 35 references that are in the map.
- First posted on arXiv 1 Mar 2023.

## Builds on (most cited first, 4 of 4)

- [modAL: A modular active learning framework for Python](https://mlsanity.com/paper/1805.00979/) (2018)
- [Streaming Active Learning Strategies for Real-Life Credit Card Fraud Detection: Assessment and Visualization](https://mlsanity.com/paper/1804.07481/) (2018)
- [Empowering Active Learning to Jointly Optimize System and User Demands](https://mlsanity.com/paper/2005.04470/) (2020)
- [Capturing Ambiguity in Crowdsourcing Frame Disambiguation](https://mlsanity.com/paper/1805.00270/) (2018)

## Built on by (most cited first, 2 of 2)

- [Recent Advancements in Machine Learning For Cybercrime Prediction](https://mlsanity.com/paper/2304.04819/) (2023)
- [Unsupervised Learning of Distributional Properties can Supplement Human Labeling and Increase Active Learning Efficiency in Anomaly Detection](https://mlsanity.com/paper/2307.08782/) (2023)

## Abstract (arXiv)

Research on email anomaly detection has typically relied on specially prepared datasets that may not adequately reflect the type of data that occurs in industry settings. In our research, at a major financial services company, privacy concerns prevented inspection of the bodies of emails and attachment details (although subject headings and attachment filenames were available). This made labeling possible anomalies in the resulting redacted emails more difficult. Another source of difficulty is the high volume of emails combined with the scarcity of resources making machine learning (ML) a necessity, but also creating a need for more efficient human training of ML models. Active learning (AL) has been proposed as a way to make human training of ML models more efficient. However, the implementation of Active Learning methods is a human-centered AI challenge due to potential human analyst uncertainty, and the labeling task can be further complicated in domains such as the cybersecurity domain (or healthcare, aviation, etc.) where mistakes in labeling can have highly adverse consequences. In this paper we present research results concerning the application of Active Learning to anomaly detection in redacted emails, comparing the utility of different methods for implementing active learning in this context. We evaluate different AL strategies and their impact on resulting model performance. We also examine how ratings of confidence that experts have in their labels can inform AL. The results obtained are discussed in terms of their implications for AL methodology and for the role of experts in model-assisted email anomaly screening.