# Online Distribution Shift Detection via Recency Prediction

arXiv: https://arxiv.org/abs/2211.09916 · HTML: https://mlsanity.com/paper/2211.09916/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 38 references that are in the map.
- First posted on arXiv 17 Nov 2022.

## Builds on (most cited first, 13 of 13)

- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Conformal prediction beyond exchangeability](https://mlsanity.com/paper/2202.13415/) (2022)
- [Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization](https://mlsanity.com/paper/2107.04649/) (2021)
- [Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift](https://mlsanity.com/paper/1810.11953/) (2018)
- [Tracking the risk of a deployed model and detecting harmful distribution shifts](https://mlsanity.com/paper/2110.06177/) (2021)
- [Testing randomness](https://mlsanity.com/paper/1906.09256/) (2019)
- [Retrain or not retrain: Conformal test martingales for change-point detection](https://mlsanity.com/paper/2102.10439/) (2021)
- [Feature Shift Detection: Localizing Which Features Have Shifted via Conditional Distribution Tests](https://mlsanity.com/paper/2107.06929/) (2021)
- [Inductive Conformal Martingales for Change-Point Detection](https://mlsanity.com/paper/1706.03415/) (2017)
- [Conformal Anomaly Detection on Spatio-Temporal Observations with Missing Data](https://mlsanity.com/paper/2105.11886/) (2021)
- [Test for non-negligible adverse shifts](https://mlsanity.com/paper/2107.02990/) (2021)

## Built on by (most cited first, 6 of 6)

- [Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progress](https://mlsanity.com/paper/2410.04640/) (2024)
- [On Continuous Monitoring of Risk Violations under Unknown Shift](https://mlsanity.com/paper/2506.16416/) (2025)
- [Continual Graph Convolutional Network for Text Classification](https://mlsanity.com/paper/2304.04152/) (2023)
- [Diagnostic Runtime Monitoring with Martingales](https://mlsanity.com/paper/2407.21748/) (2024)
- [EmbodiTTA: Resource-Efficient Test-Time Adaptation for Embodied Visual Systems](https://mlsanity.com/paper/2505.00986/) (2025)
- [Deceptive Risk Minimization: Out-of-Distribution Generalization by Deceiving Distribution Shift Detectors](https://mlsanity.com/paper/2509.12081/) (2025)

## Abstract (arXiv)

When deploying modern machine learning-enabled robotic systems in high-stakes applications, detecting distribution shift is critical. However, most existing methods for detecting distribution shift are not well-suited to robotics settings, where data often arrives in a streaming fashion and may be very high-dimensional. In this work, we present an online method for detecting distribution shift with guarantees on the false positive rate - i.e., when there is no distribution shift, our system is very unlikely (with probability < ε) to falsely issue an alert; any alerts that are issued should therefore be heeded. Our method is specifically designed for efficient detection even with high dimensional data, and it empirically achieves up to 11x faster detection on realistic robotics settings compared to prior work while maintaining a low false negative rate in practice (whenever there is a distribution shift in our experiments, our method indeed emits an alert). We demonstrate our approach in both simulation and hardware for a visual servoing task, and show that our method indeed issues an alert before a failure occurs.