mlsanityData to 7 Oct 2026

Testing for concept shift online

arXiv 2012.14246 ↗First posted 28 Dec 2020cs.LG stat.MLcited by 14

Not on the map. Why?

Abstract (arXiv)

This note continues study of exchangeability martingales, i.e., processes that are martingales under any exchangeable distribution for the observations. Such processes can be used for detecting violations of the IID assumption, which is commonly made in machine learning. Violations of the IID assumption are sometimes referred to as dataset shift, and dataset shift is sometimes subdivided into concept shift, covariate shift, etc. Our primary interest is in concept shift, but we will also discuss exchangeability martingales that decompose perfectly into two components one of which detects concept shift and the other detects what we call label shift. Our methods will be based on techniques of conformal prediction.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 3; 2023: 0; 2024: 2; 2025: 4; 2026: 1.

14 papers.

Sort:

  1. 2021Testing for Outliers with Conformal p-valuescited by 133
  2. 2022Conformal Prediction Under Feedback Covariate Shift for Biomolecular Designcited by 53
  3. 2021Tracking the risk of a deployed model and detecting harmful distribution shiftscited by 38
  4. 2024Protected Test-Time Adaptation via Online Entropy Matching: A Betting Approachcited by 22
  5. 2025WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingalescited by 13
  6. 2022Fair When Trained, Unfair When Deployed: Observable Fairness Measures are Unstable in Performative Prediction Settingscited by 11
  7. 2022A unified framework for dataset shift diagnosticscited by 7
  8. 2021Conformal testing in a binary model situationcited by 4
  9. 2026Testing For Distribution Shifts with Conditional Conformal Test Martingalescited by 3
  10. 2024Diagnostic Runtime Monitoring with Martingalescited by 2
  11. 2021Enhancement of prediction algorithms by bettingcited by 1
  12. 2025Efficient Learning Under Density Shift in Incremental Settings Using Cram'er-Rao-Based Regularizationcited by 0
2 more
  1. 2025A Conformal Predictive Measure for Assessing Catastrophic Forgettingcited by 0
  2. 2025Adapting to Fragmented and Evolving Data: A Fisher Information Perspectivecited by 0