# Pitfalls of Unlabeled Disagreement-Based Drift Detection in Streaming Tree Ensembles

arXiv: https://arxiv.org/abs/2605.12803 · HTML: https://mlsanity.com/paper/2605.12803/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 35 references that are in the map.
- First posted on arXiv 12 May 2026.

## Builds on (most cited first, 12 of 12)

- [Assessing Generalization of SGD via Disagreement](https://mlsanity.com/paper/2106.13799/) (2021)
- [Unsupervised Out-of-Distribution Detection by Maximum Classifier Discrepancy](https://mlsanity.com/paper/1908.04951/) (2019)
- [Online Deep Learning: Learning Deep Neural Networks on the Fly](https://mlsanity.com/paper/1711.03705/) (2017)
- [Agree to Disagree: Diversity through Disagreement for Better Transferability](https://mlsanity.com/paper/2202.04414/) (2022)
- [Estimating Generalization under Distribution Shifts via Domain-Invariant Representations](https://mlsanity.com/paper/2007.03511/) (2020)
- [Extremely Fast Decision Tree](https://mlsanity.com/paper/1802.08780/) (2018)
- [A Learning Based Hypothesis Test for Harmful Covariate Shift](https://mlsanity.com/paper/2212.02742/) (2022)
- [(Almost) Provable Error Bounds Under Distribution Shift via Disagreement Discrepancy](https://mlsanity.com/paper/2306.00312/) (2023)
- [DFDT: Dynamic Fast Decision Tree for IoT Data Stream Mining on Edge Devices](https://mlsanity.com/paper/2502.14011/) (2025)
- [Early Concept Drift Detection via Prediction Uncertainty](https://mlsanity.com/paper/2412.11158/) (2024)
- [Bridging Streaming Continual Learning via In-Context Large Tabular Models](https://mlsanity.com/paper/2512.11668/) (2025)
- [Instance exploitation for learning temporary concepts from sparsely labeled drifting data streams](https://mlsanity.com/paper/2009.09382/) (2020)

## Built on by (most cited first, 2 of 2)

- [Streaming Hierarchical Inference with Tabular Foundation Models](https://mlsanity.com/paper/2609.07956/) (2026)
- [Collaborative Streaming Anomaly Detection with Interactive Explanations and Ensemble Consensus](https://mlsanity.com/paper/2609.23883/) (2026)

## Abstract (arXiv)

Detecting concept drift in high-speed data streams remains challenging, particularly when models must operate on unlabeled data and avoid false alarms caused by benign shifts. While disagreement-based uncertainty has shown promise in neural networks, its adaptation to ensembles of incremental decision trees (IDTs) remains largely unexplored. We investigate this approach by constructing batch-specific disagreement measures via label flipping in ensemble members and evaluating their effectiveness for drift detection in tabular data streams. Our experiments show that, although this method performs well in ensembles of multi-layer perceptrons (MLPs), it consistently underperforms loss-based detectors when applied to IDTs. We attribute this behavior to the intrinsic rigidity of IDTs: learning primarily through structural expansion, with limited parameter adaptation, restricts model plasticity and prevents disagreement from reliably reflecting learning potential. Recent work on restructuring IDTs using their intrinsic decomposition into non-overlapping rules offers a promising direction for improving adaptability.