mlsanityData to 7 Oct 2026

Retrain or not retrain: Conformal test martingales for change-point detection

arXiv 2102.10439 ↗First posted 20 Feb 2021cs.LG stat.MLcited by 28

Not on the map. Why?

Abstract (arXiv)

We argue for supplementing the process of training a prediction algorithm by setting up a scheme for detecting the moment when the distribution of the data changes and the algorithm needs to be retrained. Our proposed schemes are based on exchangeability martingales, i.e., processes that are martingales under any exchangeable distribution for the data. Our method, based on conformal prediction, is general and can be applied on top of any modern prediction algorithm. Its validity is guaranteed, and in this paper we make first steps in exploring its efficiency.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 1 of its 15 references that are in the map. First posted on arXiv 20 Feb 2021.

Builds on

1 of 15 references.

References · 1

  1. 2019Testing randomnesscited by 32

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 1; 2023: 2; 2024: 6; 2025: 4; 2026: 9.

28 papers.

Sort:

By area:

  1. 2021Testing for Outliers with Conformal p-valuescited by 133
  2. 2021Tracking the risk of a deployed model and detecting harmful distribution shiftscited by 38
  3. 2024Protected Test-Time Adaptation via Online Entropy Matching: A Betting Approachcited by 22
  4. 2025E-valuator: Reliable Agent Verifiers with Sequential Hypothesis Testingcited by 14
  5. 2025WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingalescited by 13
  6. 2023Evaluating Machine Translation Quality with Conformal Predictive Distributionscited by 9
  7. 2025On Continuous Monitoring of Risk Violations under Unknown Shiftcited by 9
  8. 2021A unified framework for bandit multiple testingcited by 8
  9. 2022Online Distribution Shift Detection via Recency Predictioncited by 6
  10. 2021Test for non-negligible adverse shiftscited by 5
  11. 2021Conformal testing in a binary model situationcited by 4
  12. 2024Leave-One-Out-, Bootstrap- and Cross-Conformal Anomaly Detectorscited by 4
16 more
  1. 2025Conformal Prediction for Time-series Forecasting with Change Pointscited by 4
  2. 2026Testing For Distribution Shifts with Conditional Conformal Test Martingalescited by 3
  3. 2026Distribution-free changepoint localization after sequential change detectioncited by 3
  4. 2024Combining Evidence Across Filtrationscited by 2
  5. 2024Temporal Generalization Estimation in Evolving Graphscited by 2
  6. 2024Diagnostic Runtime Monitoring with Martingalescited by 2
  7. 2024Beyond Conformal Predictors: Adaptive Conformal Inference with Confidence Predictorscited by 2
  8. 2026Prediction-Powered Risk Monitoring of Deployed Models for Detecting Harmful Distribution Shiftscited by 2
  9. 2021Enhancement of prediction algorithms by bettingcited by 1
  10. 2026Conformal Anomaly Detection in Python: Moving Beyond Heuristic Thresholds with nonconformcited by 1
  11. 2026Certified Adaptive Refresh: Anytime-Valid Monitoring for Federated Conformal RAGcited by 1
  12. 2023Towards Practicable Sequential Shift Detectorscited by 0
  13. 2026Conformal Blindness: A Note on A-Cryptic change-pointscited by 0
  14. 2026Betting on Moments: Legendre Jumper Martingales for Online Exchangeability Testingcited by 0
  15. 2026When the Martingale Never Stops Firing: Anytime-Valid Gating on Real Forecast Streamscited by 0
  16. 2026Strategic Self-Consistencycited by 0