# Sequential Multivariate Change Detection with Calibrated and Memoryless False Detection Rates

arXiv: https://arxiv.org/abs/2108.00883 · HTML: https://mlsanity.com/paper/2108.00883/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 40 references that are in the map.
- First posted on arXiv 2 Aug 2021.

## Builds on (most cited first, 9 of 9)

- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [Measuring Robustness to Natural Distribution Shifts in Image Classification](https://mlsanity.com/paper/2007.00644/) (2020)
- [High-dimensional, multiscale online changepoint detection](https://mlsanity.com/paper/2003.03668/) (2020)
- [Change Detection with the Kernel Cumulative Sum Algorithm](https://mlsanity.com/paper/1903.01661/) (2019)
- [Sequential change-point detection based on nearest neighbors](https://mlsanity.com/paper/1604.03611/) (2016)
- [Real-Time Nonparametric Anomaly Detection in High-Dimensional Settings](https://mlsanity.com/paper/1809.05250/) (2018)
- [Online Multivariate Anomaly Detection and Localization for High-dimensional Settings](https://mlsanity.com/paper/1905.07107/) (2019)

## Built on by (most cited first, 5 of 5)

- [Context-Aware Drift Detection](https://mlsanity.com/paper/2203.08644/) (2022)
- [Online Kernel CUSUM for Change-Point Detection](https://mlsanity.com/paper/2211.15070/) (2022)
- [Quickest Change Detection for Unnormalized Statistical Models](https://mlsanity.com/paper/2302.00250/) (2023)
- [Towards Practicable Sequential Shift Detectors](https://mlsanity.com/paper/2307.14758/) (2023)
- [Asymptotically Optimal Change Detection for Unnormalized Pre- and Post-Change Distributions](https://mlsanity.com/paper/2410.14615/) (2024)

## Abstract (arXiv)

Responding appropriately to the detections of a sequential change detector requires knowledge of the rate at which false positives occur in the absence of change. Setting detection thresholds to achieve a desired false positive rate is challenging. Existing works resort to setting time-invariant thresholds that focus on the expected runtime of the detector in the absence of change, either bounding it loosely from below or targeting it directly but with asymptotic arguments that we show cause significant miscalibration in practice. We present a simulation-based approach to setting time-varying thresholds that allows a desired expected runtime to be accurately targeted whilst additionally keeping the false positive rate constant across time steps. Whilst the approach to threshold setting is metric agnostic, we show how the cost of using the popular quadratic time MMD estimator can be reduced from O(N2B) to O(N2+NB) during configuration and from O(N2) to O(N) during operation, where N and B are the numbers of reference and bootstrap samples respectively.