# Maintaining IoT Device Identification under Concept Drift via Budget-Aware Traffic Labeling

arXiv: https://arxiv.org/abs/2608.15465 · HTML: https://mlsanity.com/paper/2608.15465/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 84 references that are in the map.
- First posted on arXiv 16 Aug 2026.

## Builds on (most cited first, 30 of 30)

- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [MADE: Masked Autoencoder for Distribution Estimation](https://mlsanity.com/paper/1502.03509/) (2015)
- [Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift](https://mlsanity.com/paper/1810.11953/) (2018)
- [State-Relabeling Adversarial Active Learning](https://mlsanity.com/paper/2004.04943/) (2020)
- [Transcending Transcend: Revisiting Malware Classification in the Presence of Concept Drift](https://mlsanity.com/paper/2010.03856/) (2020)
- [Continuous Learning for Android Malware Detection](https://mlsanity.com/paper/2302.04332/) (2023)
- [IoT Sentinel: Automated Device-Type Identification for Security Enforcement in IoT](https://mlsanity.com/paper/1611.04880/) (2016)
- [Machine Learning DDoS Detection for Consumer Internet of Things Devices](https://mlsanity.com/paper/1804.04159/) (2018)
- [STUDD: A Student-Teacher Method for Unsupervised Concept Drift Detection](https://mlsanity.com/paper/2103.00903/) (2021)
- [PWPAE: An Ensemble Framework for Concept Drift Adaptation in IoT Data Streams](https://mlsanity.com/paper/2109.05013/) (2021)
- [Autonomous Unknown-Application Filtering and Labeling for DL-based Traffic Classifier Update](https://mlsanity.com/paper/2002.06359/) (2020)
- [Datasets are not Enough: Challenges in Labeling Network Traffic](https://mlsanity.com/paper/2110.05977/) (2021)
- [Low-Quality Training Data Only? A Robust Framework for Detecting Encrypted Malicious Network Traffic](https://mlsanity.com/paper/2309.04798/) (2023)
- [Revisiting Concept Drift in Windows Malware Detection: Adaptation to Real Drifted Malware with Minimal Samples](https://mlsanity.com/paper/2407.13918/) (2024)
- [LEAF: Navigating Concept Drift in Cellular Networks](https://mlsanity.com/paper/2109.03011/) (2021)
- [Extensible Machine Learning for Encrypted Network Traffic Application Labeling via Uncertainty Quantification](https://mlsanity.com/paper/2205.05628/) (2022)
- [Peek-a-Boo: I see your smart home activities, even encrypted!](https://mlsanity.com/paper/1808.02741/) (2018)
- [When to retrain a machine learning model](https://mlsanity.com/paper/2505.14903/) (2025)
- [Tracking, Profiling, and Ad Targeting in the Alexa Echo Smart Speaker Ecosystem](https://mlsanity.com/paper/2204.10920/) (2022)
- [A Smart Home is No Castle: Privacy Vulnerabilities of Encrypted IoT Traffic](https://mlsanity.com/paper/1705.06805/) (2017)
- [A Haystack Full of Needles: Scalable Detection of IoT Devices in the Wild](https://mlsanity.com/paper/2009.01880/) (2020)
- [The Case for Retraining of ML Models for IoT Device Identification at the Edge](https://mlsanity.com/paper/2011.08605/) (2020)
- [Revisiting IoT Device Identification](https://mlsanity.com/paper/2107.07818/) (2021)
- [Verifying and Monitoring IoTs Network Behavior using MUD Profiles](https://mlsanity.com/paper/1902.02484/) (2019)
- [Blocking without Breaking: Identification and Mitigation of Non-Essential IoT Traffic](https://mlsanity.com/paper/2105.05162/) (2021)
- [Characterizing User Platforms for Video Streaming in Broadband Networks](https://mlsanity.com/paper/2408.16995/) (2024)
- [Sensitive Information Tracking in Commodity IoT](https://mlsanity.com/paper/1802.08307/) (2018)
- [WiFinger: Fingerprinting Noisy IoT Event Traffic Using Packet-level Sequence Matching](https://mlsanity.com/paper/2508.03151/) (2025)
- [Binary Anomaly Detection in Streaming IoT Traffic under Concept Drift](https://mlsanity.com/paper/2510.27304/) (2025)
- [EMA: Efficient Model Adaptation for Learning-based Systems](https://mlsanity.com/paper/2605.13942/) (2026)

## Abstract (arXiv)

Identification of IoT device types from passive traffic is increasingly used for security management in enterprise and ISP networks. However, the performance of machine learning-based classifiers gradually degrades under concept drift as device behavior evolves. Therefore, maintaining classification performance requires periodic retraining with newly labeled deployment traffic. The operational challenge is determining how much and which deployment traffic instances to label for maintaining classification performance. We show that these two decisions should be treated separately. While retraining solely on instances selected by a drift detector is prone to systematically overlooking parts of the emerging behavioral space, uniformly sampled deployment traffic captures more representative behavioral changes. Instead, drift detection is more effective at determining the amount of deployment traffic that should be labeled. We make three contributions. (1) We conduct a two-year longitudinal study of IoT traffic and characterize how behavioral evolution manifests across device classes and how retraining with newly labeled traffic restores classification performance. (2) We develop a conformity-based drift detector that captures class-conditional behavioral models directly from raw traffic features and provides feature-level explanations of behavioral evolution. (3) We demonstrate that adjusting the traffic labeling rate according to the observed behavioral evolution, combined with uniform traffic sampling, maintains classifier performance more effectively than detector-guided sample selection and is beneficial to managing the traffic labeling effort. We further show that this strategy performs comparably to confidence-guided adaptation while providing feature-level explanations. Our evaluation uses 3.8 million IPFIX flow records collected from 21 IoT types over more than 2 years.