# An Automated Data Engineering Pipeline for Anomaly Detection of IoT Sensor Data

arXiv: https://arxiv.org/abs/2109.13828 · HTML: https://mlsanity.com/paper/2109.13828/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 35 references that are in the map.
- First posted on arXiv 28 Sep 2021.

## Builds on (most cited first, 12 of 12)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice](https://mlsanity.com/paper/2007.15745/) (2020)
- [k-Nearest Neighbour Classifiers: 2nd Edition (with Python examples)](https://mlsanity.com/paper/2004.04523/) (2020)
- [MTH-IDS: A Multi-Tiered Hybrid Intrusion Detection System for Internet of Vehicles](https://mlsanity.com/paper/2105.13289/) (2021)
- [A Lightweight Concept Drift Detection and Adaptation Framework for IoT Data Streams](https://mlsanity.com/paper/2104.10529/) (2021)
- [Tree-based Intelligent Intrusion Detection System in Internet of Vehicles](https://mlsanity.com/paper/1910.08635/) (2019)
- [Machine Learning Towards Intelligent Systems: Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2101.03655/) (2021)
- [Trends in Development of Databases and Blockchain](https://mlsanity.com/paper/2003.05687/) (2020)
- [Detecting Botnet Attacks in IoT Environments: An Optimized Machine Learning Approach](https://mlsanity.com/paper/2012.11325/) (2020)
- [Multi-Perspective Content Delivery Networks Security Framework Using Optimized Unsupervised Anomaly Detection](https://mlsanity.com/paper/2107.11514/) (2021)
- [A Transfer Learning Framework for Anomaly Detection Using Model of Normality](https://mlsanity.com/paper/2011.06210/) (2020)
- [DNS Typo-squatting Domain Detection: A Data Analytics & Machine Learning Based Approach](https://mlsanity.com/paper/2012.13604/) (2020)

## Abstract (arXiv)

The rapid development in the field of System of Chip (SoC) technology, Internet of Things (IoT), cloud computing, and artificial intelligence has brought more possibilities of improving and solving the current problems. With data analytics and the use of machine learning/deep learning, it is made possible to learn the underlying patterns and make decisions based on what was learned from massive data generated from IoT sensors. When combined with cloud computing, the whole pipeline can be automated, and free of manual controls and operations. In this paper, an implementation of an automated data engineering pipeline for anomaly detection of IoT sensor data is studied and proposed. The process involves the use of IoT sensors, Raspberry Pis, Amazon Web Services (AWS) and multiple machine learning techniques with the intent to identify anomalous cases for the smart home security system.