mlsanityData to 7 Oct 2026

Ensembling Shift Detectors: an Extensive Empirical Evaluation

arXiv 2106.14608 ↗First posted 28 Jun 2021cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

The term dataset shift refers to the situation where the data used to train a machine learning model is different from where the model operates. While several types of shifts naturally occur, existing shift detectors are usually designed to address only a specific type of shift. We propose a simple yet powerful technique to ensemble complementary shift detectors, while tuning the significance level of each detector's statistical test to the dataset. This enables a more robust shift detection, capable of addressing all different types of shift, which is essential in real-life settings where the precise shift type is often unknown. This approach is validated by a large-scale statistically sound benchmark study over various synthetic shifts applied to real-world structured datasets.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 12 of its 28 references that are in the map. First posted on arXiv 28 Jun 2021.

Builds on

12 of 28 references.

References · 12

  1. 2016Concrete Problems in AI Safetycited by 1,582
  2. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  3. 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 656
  4. 2017Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Modelscited by 459
  5. 2020A Unifying Review of Deep and Shallow Anomaly Detectioncited by 423
  6. 2018Detecting and Correcting for Label Shift with Black Box Predictorscited by 364
  7. 2018Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shiftcited by 240
  8. 2019A review of domain adaptation without target labelscited by 154
  9. 2020A Unified View of Label Shift Estimationcited by 119
  10. 2019Regularized Learning for Domain Adaptation under Label Shiftscited by 118
  11. 2020A survey on domain adaptation theory: learning bounds and theoretical guaranteescited by 109
  12. 2020Monitoring and explainability of models in productioncited by 23

Built on by

Cited by year

Citing papers per year: 2022: 2.

2 papers.

  1. 2022Performance Prediction Under Dataset Shiftcited by 3
  2. 2022Towards Clear Expectations for Uncertainty Estimationcited by 2