# A Scalable Approach to Covariate and Concept Drift Management via Adaptive Data Segmentation

arXiv: https://arxiv.org/abs/2411.15616 · HTML: https://mlsanity.com/paper/2411.15616/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 47 references that are in the map.
- First posted on arXiv 23 Nov 2024.

## Builds on (most cited first, 10 of 10)

- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Training](https://mlsanity.com/paper/2103.00123/) (2021)
- [GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learning](https://mlsanity.com/paper/2012.10630/) (2020)
- [Characterizing Concept Drift](https://mlsanity.com/paper/1511.03816/) (2015)
- [The Random Forest Kernel and other kernels for big data from random partitions](https://mlsanity.com/paper/1402.4293/) (2014)
- [Concept Drift Adaptation by Exploiting Historical Knowledge](https://mlsanity.com/paper/1702.03500/) (2017)
- [Handling Concept Drift via Model Reuse](https://mlsanity.com/paper/1809.02804/) (2018)
- [How good is the Electricity benchmark for evaluating concept drift adaptation](https://mlsanity.com/paper/1301.3524/) (2013)
- [Quilt: Robust Data Segment Selection against Concept Drifts](https://mlsanity.com/paper/2312.09691/) (2023)
- [DriftSurf: A Risk-competitive Learning Algorithm under Concept Drift](https://mlsanity.com/paper/2003.06508/) (2020)

## Built on by (most cited first, 2 of 2)

- [CRUMB: Efficient Prior Fitted Network Inference via Distributionally Matched Context Batching](https://mlsanity.com/paper/2606.11473/) (2026)
- [FMMO: Detecting the Divergence Between Local Attribution and Global Drift](https://mlsanity.com/paper/2609.06173/) (2026)

## Abstract (arXiv)

In many real-world applications, continuous machine learning (ML) systems are crucial but prone to data drift, a phenomenon where discrepancies between historical training data and future test data lead to significant performance degradation and operational inefficiencies. Traditional drift adaptation methods typically update models using ensemble techniques, often discarding drifted historical data, and focus primarily on either covariate drift or concept drift. These methods face issues such as high resource demands, inability to manage all types of drifts effectively, and neglecting the valuable context that historical data can provide. We contend that explicitly incorporating drifted data into the model training process significantly enhances model accuracy and robustness. This paper introduces an advanced framework that integrates the strengths of data-centric approaches with adaptive management of both covariate and concept drift in a scalable and efficient manner. Our framework employs sophisticated data segmentation techniques to identify optimal data batches that accurately reflect test data patterns. These data batches are then utilized for training on test data, ensuring that the models remain relevant and accurate over time. By leveraging the advantages of both data segmentation and scalable drift management, our solution ensures robust model accuracy and operational efficiency in large-scale ML deployments. It also minimizes resource consumption and computational overhead by selecting and utilizing relevant data subsets, leading to significant cost savings. Experimental results on classification task on real-world and synthetic datasets show our approach improves model accuracy while reducing operational costs and latency. This practical solution overcomes inefficiencies in current methods, providing a robust, adaptable, and scalable approach.