A Lightweight Concept Drift Detection and Adaptation Framework for IoT Data Streams
Not on the map. Why?
Abstract (arXiv)
In recent years, with the increasing popularity of "Smart Technology", the number of Internet of Things (IoT) devices and systems have surged significantly. Various IoT services and functionalities are based on the analytics of IoT streaming data. However, IoT data analytics faces concept drift challenges due to the dynamic nature of IoT systems and the ever-changing patterns of IoT data streams. In this article, we propose an adaptive IoT streaming data analytics framework for anomaly detection use cases based on optimized LightGBM and concept drift adaptation. A novel drift adaptation method named Optimized Adaptive and Sliding Windowing (OASW) is proposed to adapt to the pattern changes of online IoT data streams. Experiments on two public datasets show the high accuracy and efficiency of our proposed adaptive LightGBM model compared against other state-of-the-art approaches. The proposed adaptive LightGBM model can perform continuous learning and drift adaptation on IoT data streams without human intervention.
Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 6 of its 19 references that are in the map. First posted on arXiv 21 Apr 2021.
Builds on
6 of 19 references.
References · 6
- 2020Learning under Concept Drift: A Reviewcited by 470
- 2020On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practicecited by 181
- 2018Scikit-Multiflow: A Multi-output Streaming Frameworkcited by 70
- 2018Extremely Fast Decision Treecited by 42
- 2019Tree-based Intelligent Intrusion Detection System in Internet of Vehiclescited by 25
- 2020Detecting Botnet Attacks in IoT Environments: An Optimized Machine Learning Approachcited by 9
Built on by
Citing papers per year: 2021: 6; 2022: 9; 2023: 3; 2024: 1; 2025: 7; 2026: 2.
28 papers.
Sort:
- 2021MTH-IDS: A Multi-Tiered Hybrid Intrusion Detection System for Internet of Vehiclescited by 36
- 2021PWPAE: An Ensemble Framework for Concept Drift Adaptation in IoT Data Streamscited by 17
- 2022IoT Data Analytics in Dynamic Environments: From An Automated Machine Learning Perspectivecited by 15
- 2022A Transfer Learning and Optimized CNN Based Intrusion Detection System for Internet of Vehiclescited by 11
- 2021Concept Drift Detection in Federated Networked Systemscited by 10
- 2021Multi-Perspective Content Delivery Networks Security Framework Using Optimized Unsupervised Anomaly Detectioncited by 9
- 2022Online Federated Learning via Non-Stationary Detection and Adaptation amidst Concept Driftcited by 9
- 2022LCCDE: A Decision-Based Ensemble Framework for Intrusion Detection in The Internet of Vehiclescited by 6
- 2025Enabling AutoML for Zero-Touch Network Security: Use-Case Driven Analysiscited by 4
- 2023Fast Context Adaptation in Cost-Aware Continual Learningcited by 2
- 2025A Transfer Learning Framework for Anomaly Detection in Multivariate IoT Traffic Datacited by 2
- 2025Towards Zero Touch Networks: Cross-Layer Automated Security Solutions for 6G Wireless Networkscited by 2
16 more
- 2025RCCDA: Adaptive Model Updates in the Presence of Concept Drift under a Constrained Resource Budgetcited by 2
- 2022A Model Drift Detection and Adaptation Framework for 5G Core Networkscited by 1
- 2022Addressing Data Distribution Shifts in Online Machine Learning Powered Smart City Applications Using Augmented Test-Time Adaptationcited by 1
- 2024An incremental hybrid adaptive network-based IDS in Software Defined Networks to detect stealth attackscited by 1
- 2025Toward Autonomous and Efficient Cybersecurity: A Multi-Objective AutoML-based Intrusion Detection Systemcited by 1
- 2021Assessing Machine Learning Approaches to Address IoT Sensor Driftcited by 0
- 2021An Automated Data Engineering Pipeline for Anomaly Detection of IoT Sensor Datacited by 0
- 2022Anomaly Detection in Intra-Vehicle Networkscited by 0
- 2022An Edge-Cloud Integrated Framework for Flexible and Dynamic Stream Analyticscited by 0
- 2022A Sequential Concept Drift Detection Method for On-Device Learning on Low-End Edge Devicescited by 0
- 2023Modular Simulation Environment Towards OTN AI-based Solutionscited by 0
- 2023Forensic Data Analytics for Anomaly Detection in Evolving Networkscited by 0
- 2025Data-Driven Heat Pump Management: Combining Machine Learning with Anomaly Detection for Residential Hot Water Systemscited by 0
- 2025Uncertainty Guided Online Ensemble for Non-stationary Data Streams in Fusion Sciencecited by 0
- 2026A Continual Validation, Updating, and Decision-Making Framework for Self-Adaptive Digital Twins via Robust Model Predictive Control: A Case Study in Additive Manufacturingcited by 0
- 2026Performance Drift Detection in Machine Learning as a Service (MLaaS) for IoT Environmentscited by 0
- 2026Performance Drift Detection in Machine Learning as a Service (MLaaS) for IoT Environments19 Aug 2026
- 2026A Continual Validation, Updating, and Decision-Making Framework for Self-Adaptive Digital Twins via Robust Model Predictive Control: A Case Study in Additive Manufacturing20 Jul 2026
- 2025Toward Autonomous and Efficient Cybersecurity: A Multi-Objective AutoML-based Intrusion Detection System11 Nov 2025
- 2025Uncertainty Guided Online Ensemble for Non-stationary Data Streams in Fusion Science3 Nov 2025
- 2025Data-Driven Heat Pump Management: Combining Machine Learning with Anomaly Detection for Residential Hot Water Systems3 Jun 2025
- 2025RCCDA: Adaptive Model Updates in the Presence of Concept Drift under a Constrained Resource Budget30 May 2025
- 2025Enabling AutoML for Zero-Touch Network Security: Use-Case Driven Analysis28 Feb 2025
- 2025Towards Zero Touch Networks: Cross-Layer Automated Security Solutions for 6G Wireless Networks28 Feb 2025
- 2025A Transfer Learning Framework for Anomaly Detection in Multivariate IoT Traffic Data26 Jan 2025
- 2024An incremental hybrid adaptive network-based IDS in Software Defined Networks to detect stealth attacks1 Apr 2024
- 2023Forensic Data Analytics for Anomaly Detection in Evolving Networks17 Aug 2023
- 2023Modular Simulation Environment Towards OTN AI-based Solutions19 Jun 2023
16 more
- 2023Fast Context Adaptation in Cost-Aware Continual Learning6 Jun 2023
- 2022A Sequential Concept Drift Detection Method for On-Device Learning on Low-End Edge Devices19 Dec 2022
- 2022Online Federated Learning via Non-Stationary Detection and Adaptation amidst Concept Drift22 Nov 2022
- 2022Addressing Data Distribution Shifts in Online Machine Learning Powered Smart City Applications Using Augmented Test-Time Adaptation2 Nov 2022
- 2022IoT Data Analytics in Dynamic Environments: From An Automated Machine Learning Perspective16 Sep 2022
- 2022A Model Drift Detection and Adaptation Framework for 5G Core Networks8 Aug 2022
- 2022LCCDE: A Decision-Based Ensemble Framework for Intrusion Detection in The Internet of Vehicles5 Aug 2022
- 2022An Edge-Cloud Integrated Framework for Flexible and Dynamic Stream Analytics10 May 2022
- 2022Anomaly Detection in Intra-Vehicle Networks7 May 2022
- 2022A Transfer Learning and Optimized CNN Based Intrusion Detection System for Internet of Vehicles27 Jan 2022
- 2021An Automated Data Engineering Pipeline for Anomaly Detection of IoT Sensor Data28 Sep 2021
- 2021Concept Drift Detection in Federated Networked Systems13 Sep 2021
- 2021PWPAE: An Ensemble Framework for Concept Drift Adaptation in IoT Data Streams10 Sep 2021
- 2021Assessing Machine Learning Approaches to Address IoT Sensor Drift2 Sep 2021
- 2021Multi-Perspective Content Delivery Networks Security Framework Using Optimized Unsupervised Anomaly Detection24 Jul 2021
- 2021MTH-IDS: A Multi-Tiered Hybrid Intrusion Detection System for Internet of Vehicles26 May 2021
- 2021MTH-IDS: A Multi-Tiered Hybrid Intrusion Detection System for Internet of Vehiclescited by 36
- 2021PWPAE: An Ensemble Framework for Concept Drift Adaptation in IoT Data Streamscited by 17
- 2022IoT Data Analytics in Dynamic Environments: From An Automated Machine Learning Perspectivecited by 15
- 2022A Transfer Learning and Optimized CNN Based Intrusion Detection System for Internet of Vehiclescited by 11
- 2021Concept Drift Detection in Federated Networked Systemscited by 10
- 2021Multi-Perspective Content Delivery Networks Security Framework Using Optimized Unsupervised Anomaly Detectioncited by 9
- 2022Online Federated Learning via Non-Stationary Detection and Adaptation amidst Concept Driftcited by 9
- 2022LCCDE: A Decision-Based Ensemble Framework for Intrusion Detection in The Internet of Vehiclescited by 6
- 2025Enabling AutoML for Zero-Touch Network Security: Use-Case Driven Analysiscited by 4
- 2023Fast Context Adaptation in Cost-Aware Continual Learningcited by 2
- 2025A Transfer Learning Framework for Anomaly Detection in Multivariate IoT Traffic Datacited by 2
- 2025Towards Zero Touch Networks: Cross-Layer Automated Security Solutions for 6G Wireless Networkscited by 2
12 most cited of 28 All 28 (JSON)
- 2026Performance Drift Detection in Machine Learning as a Service (MLaaS) for IoT Environments19 Aug 2026
- 2026A Continual Validation, Updating, and Decision-Making Framework for Self-Adaptive Digital Twins via Robust Model Predictive Control: A Case Study in Additive Manufacturing20 Jul 2026
- 2025Toward Autonomous and Efficient Cybersecurity: A Multi-Objective AutoML-based Intrusion Detection System11 Nov 2025
- 2025Uncertainty Guided Online Ensemble for Non-stationary Data Streams in Fusion Science3 Nov 2025
- 2025Data-Driven Heat Pump Management: Combining Machine Learning with Anomaly Detection for Residential Hot Water Systems3 Jun 2025
- 2025RCCDA: Adaptive Model Updates in the Presence of Concept Drift under a Constrained Resource Budget30 May 2025
- 2025Enabling AutoML for Zero-Touch Network Security: Use-Case Driven Analysis28 Feb 2025
- 2025Towards Zero Touch Networks: Cross-Layer Automated Security Solutions for 6G Wireless Networks28 Feb 2025
- 2025A Transfer Learning Framework for Anomaly Detection in Multivariate IoT Traffic Data26 Jan 2025
- 2024An incremental hybrid adaptive network-based IDS in Software Defined Networks to detect stealth attacks1 Apr 2024
- 2023Forensic Data Analytics for Anomaly Detection in Evolving Networks17 Aug 2023
- 2023Modular Simulation Environment Towards OTN AI-based Solutions19 Jun 2023
12 newest of 28 All 28 (JSON)