# Fairness Hub Technical Briefs: Definition and Detection of Distribution Shift

arXiv: https://arxiv.org/abs/2405.14186 · HTML: https://mlsanity.com/paper/2405.14186/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 20 references that are in the map.
- First posted on arXiv 23 May 2024.

## Builds on (most cited first, 8 of 8)

- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [Improving robustness against common corruptions by covariate shift adaptation](https://mlsanity.com/paper/2006.16971/) (2020)
- [Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift](https://mlsanity.com/paper/1810.11953/) (2018)
- [Learning Deep Kernels for Non-Parametric Two-Sample Tests](https://mlsanity.com/paper/2002.09116/) (2020)
- [From Concept Drift to Model Degradation: An Overview on Performance-Aware Drift Detectors](https://mlsanity.com/paper/2203.11070/) (2022)
- [Online Adaptation to Label Distribution Shift](https://mlsanity.com/paper/2107.04520/) (2021)
- [Diagnosing Model Performance Under Distribution Shift](https://mlsanity.com/paper/2303.02011/) (2023)
- [Concept Drift Detection for Streaming Data](https://mlsanity.com/paper/1504.01044/) (2015)

## Abstract (arXiv)

Distribution shift is a common situation in machine learning tasks, where the data used for training a model is different from the data the model is applied to in the real world. This issue arises across multiple technical settings: from standard prediction tasks, to time-series forecasting, and to more recent applications of large language models (LLMs). This mismatch can lead to performance reductions, and can be related to a multiplicity of factors: sampling issues and non-representative data, changes in the environment or policies, or the emergence of previously unseen scenarios. This brief focuses on the definition and detection of distribution shifts in educational settings. We focus on standard prediction problems, where the task is to learn a model that takes in a series of input (predictors) X=(x1,x2,...,xm) and produces an output Y=f(X).