# Extending Isolation Forest for Anomaly Detection in Big Data via K-Means

arXiv: https://arxiv.org/abs/2104.13190 · HTML: https://mlsanity.com/paper/2104.13190/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 94 references that are in the map.
- First posted on arXiv 27 Apr 2021.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Deep Learning for Anomaly Detection: A Survey](https://mlsanity.com/paper/1901.03407/) (2019)
- [Extended Isolation Forest](https://mlsanity.com/paper/1811.02141/) (2018)
- [MLlib: Machine Learning in Apache Spark](https://mlsanity.com/paper/1505.06807/) (2015)
- [Unsupervised and Semi-supervised Anomaly Detection with LSTM Neural Networks](https://mlsanity.com/paper/1710.09207/) (2017)
- [WSL-DS: Weakly Supervised Learning with Distant Supervision for Query Focused Multi-Document Abstractive Summarization](https://mlsanity.com/paper/2011.01421/) (2020)
- [Zero-Resource Cross-Lingual Named Entity Recognition](https://mlsanity.com/paper/1911.09812/) (2019)
- [Distortion Robust Image Classification using Deep Convolutional Neural Network with Discrete Cosine Transform](https://mlsanity.com/paper/1811.05819/) (2018)
- [Utilizing Bidirectional Encoder Representations from Transformers for Answer Selection](https://mlsanity.com/paper/2011.07208/) (2020)
- [HealthGuard: A Machine Learning-Based Security Framework for Smart Healthcare Systems](https://mlsanity.com/paper/1909.10565/) (2019)
- [A Comparative Study of AI-based Intrusion Detection Techniques in Critical Infrastructures](https://mlsanity.com/paper/2008.00088/) (2020)
- [Switching and Data Injection Attacks on Stochastic Cyber-Physical Systems: Modeling, Resilient Estimation and Attack Mitigation](https://mlsanity.com/paper/1707.07112/) (2017)
- [ClustCrypt: Privacy-Preserving Clustering of Unstructured Big Data in the Cloud](https://mlsanity.com/paper/1908.04960/) (2019)

## Built on by (most cited first, 3 of 3)

- [Anomaly Detection Based on Isolation Mechanisms: A Survey](https://mlsanity.com/paper/2403.10802/) (2024)
- [Boosting Anomaly Detection Using Unsupervised Diverse Test-Time Augmentation](https://mlsanity.com/paper/2110.15700/) (2021)
- [CatBack: Universal Backdoor Attacks on Tabular Data via Categorical Encoding](https://mlsanity.com/paper/2511.06072/) (2025)

## Abstract (arXiv)

Industrial Information Technology (IT) infrastructures are often vulnerable to cyberattacks. To ensure security to the computer systems in an industrial environment, it is required to build effective intrusion detection systems to monitor the cyber-physical systems (e.g., computer networks) in the industry for malicious activities. This paper aims to build such intrusion detection systems to protect the computer networks from cyberattacks. More specifically, we propose a novel unsupervised machine learning approach that combines the K-Means algorithm with the Isolation Forest for anomaly detection in industrial big data scenarios. Since our objective is to build the intrusion detection system for the big data scenario in the industrial domain, we utilize the Apache Spark framework to implement our proposed model which was trained in large network traffic data (about 123 million instances of network traffic) stored in Elasticsearch. Moreover, we evaluate our proposed model on the live streaming data and find that our proposed system can be used for real-time anomaly detection in the industrial setup. In addition, we address different challenges that we face while training our model on large datasets and explicitly describe how these issues were resolved. Based on our empirical evaluation in different use-cases for anomaly detection in real-world network traffic data, we observe that our proposed system is effective to detect anomalies in big data scenarios. Finally, we evaluate our proposed model on several academic datasets to compare with other models and find that it provides comparable performance with other state-of-the-art approaches.