mlsanityData to 14 Oct 2026

Dimension-Calibrated Unexplained Mass: An Interpretable Drift Statistic for Contamination Monitoring in Data Streams

arXiv 2607.16811 ↗First posted 18 Jul 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Drift detectors that work tend not to explain themselves, and drift detectors that explain themselves tend to fail in high dimension. We close that gap for Gaussian mixture models (GMMs): each fitted component is a named "regime," and the fraction of a stream window matching no regime -- its unexplained mass -- is a drift signal that is simultaneously its own explanation. We identify why this statistic collapses in high dimension and repair it. Under a correct component a normal point in d dimensions lies about sqrt(d) sigma from the mean, so once d exceeds 9 essentially every point exceeds a fixed 3-sigma radius: window-level ROC-AUC is exactly 0.50 on Satellite (d=36) and Optdigits (d=64). Calibrating the radius to sqrt(chi-squared_d(0.99)) removes the collapse -- AUC 1.00 and 0.89 -- while leaving low dimensions unchanged. Across seven public benchmarks, five seeds, and eight model-free detectors spanning the kernel, classifier, projection, density-difference, transport, likelihood and partition families, the repaired statistic is best or tied-best on five of seven datasets at 10% window contamination (its two losses are Pendigits, where the whole field beats it, and Optdigits), and as contamination becomes sparse the sample-level detectors fade toward chance while it degrades most gracefully: at 2% its mean AUC across the benchmarks is 0.86 against at most 0.73 for any model-free detector (1.00 vs. MMD's 0.72 on KDD-http) -- while alone among them reporting which regime the data left and how far outside it the window lies. We delimit its scope honestly: unexplained mass detects and explains novel-regime drift but is blind by construction to in-support re-weighting of known regimes, where distribution-level tests are required and explain nothing; and the underlying density model's EVT-calibrated false-alarm rates degrade above d of about 36. All code and experiments are released.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 58 references that are in the map. First posted on arXiv 18 Jul 2026.

Builds on

17 of 58 references.

References · 17

  1. 2020Learning under Concept Drift: A Reviewcited by 477
  2. 2016Revisiting Classifier Two-Sample Testscited by 268
  3. 2018Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shiftcited by 243
  4. 2022ADBench: Anomaly Detection Benchmarkcited by 232
  5. 2019PyOD: A Python Toolbox for Scalable Outlier Detectioncited by 222
  6. 2022ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functionscited by 153
  7. 2020COPOD: Copula-Based Outlier Detectioncited by 146
  8. 2020Learning Deep Kernels for Non-Parametric Two-Sample Testscited by 130
  9. 2021MMD Aggregated Two-Sample Testcited by 75
  10. 2023Model Based Explanations of Concept Driftcited by 21
  11. 2022AutoML Two-Sample Testcited by 20
  12. 2021STUDD: A Student-Teacher Method for Unsupervised Concept Drift Detectioncited by 19
5 more
  1. 2024Unsupervised Concept Drift Detection from Deep Learning Representations in Real-timecited by 16
  2. 2024Online Drift Detection with Maximum Concept Discrepancycited by 12
  3. 2020Concept Drift Detection via Equal Intensity k-means Space Partitioningcited by 8
  4. 2022Change Detection for Local Explainability in Evolving Data Streamscited by 8
  5. 2022Nonparametric and Online Change Detection in Multivariate Datastreams using QuantTreecited by 6

Built on by

Not cited yet.