# Feature Shift Detection: Localizing Which Features Have Shifted via Conditional Distribution Tests

arXiv: https://arxiv.org/abs/2107.06929 · HTML: https://mlsanity.com/paper/2107.06929/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 14 Jul 2021.

## Builds on (most cited first, 14 of 14)

- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift](https://mlsanity.com/paper/1810.11953/) (2018)
- [Adversarially Learned Anomaly Detection](https://mlsanity.com/paper/1812.02288/) (2018)
- [Deep AutoRegressive Networks](https://mlsanity.com/paper/1310.8499/) (2013)
- [Iterative Gaussianization: from ICA to Random Rotations](https://mlsanity.com/paper/1602.00229/) (2016)
- [Variational approximations using Fisher divergence](https://mlsanity.com/paper/1905.05284/) (2019)
- [Request-and-Reverify: Hierarchical Hypothesis Testing for Concept Drift Detection with Expensive Labels](https://mlsanity.com/paper/1806.10131/) (2018)
- [Fisher Efficient Inference of Intractable Models](https://mlsanity.com/paper/1805.07454/) (2018)
- [Online Decorrelation of Humidity and Temperature in Chemical Sensors for Continuous Monitoring](https://mlsanity.com/paper/1608.01719/) (2016)
- [Diverse Instances-Weighting Ensemble based on Region Drift Disagreement for Concept Drift Adaptation](https://mlsanity.com/paper/2004.05810/) (2020)

## Built on by (most cited first, 24 of 24)

- [Fairness Transferability Subject to Bounded Distribution Shift](https://mlsanity.com/paper/2206.00129/) (2022)
- [Towards Explaining Distribution Shifts](https://mlsanity.com/paper/2210.10275/) (2022)
- [Calibration of Time-Series Forecasting: Detecting and Adapting Context-Driven Distribution Shift](https://mlsanity.com/paper/2310.14838/) (2023)
- ["Why did the Model Fail?": Attributing Model Performance Changes to Distribution Shifts](https://mlsanity.com/paper/2210.10769/) (2022)
- [Out-of-Distribution Generalization in Time Series: A Survey](https://mlsanity.com/paper/2503.13868/) (2025)
- [Online Distribution Shift Detection via Recency Prediction](https://mlsanity.com/paper/2211.09916/) (2022)
- [A hierarchical decomposition for explaining ML performance discrepancies](https://mlsanity.com/paper/2402.14254/) (2024)
- [Inherent Diverse Redundant Safety Mechanisms for AI-based Software Elements in Automotive Applications](https://mlsanity.com/paper/2402.08208/) (2024)
- [Identifying General Mechanism Shifts in Linear Causal Representations](https://mlsanity.com/paper/2410.24059/) (2024)
- [Diagnostic Runtime Monitoring with Martingales](https://mlsanity.com/paper/2407.21748/) (2024)
- [MLHOps: Machine Learning for Healthcare Operations](https://mlsanity.com/paper/2305.02474/) (2023)
- [Adversarial Learning for Feature Shift Detection and Correction](https://mlsanity.com/paper/2312.04546/) (2023)
- [Deephys: Deep Electrophysiology, Debugging Neural Networks under Distribution Shifts](https://mlsanity.com/paper/2303.11912/) (2023)
- [Interpretable Model Drift Detection](https://mlsanity.com/paper/2503.06606/) (2025)
- ["Who experiences large model decay and why?" A Hierarchical Framework for Diagnosing Heterogeneous Performance Drift](https://mlsanity.com/paper/2506.00756/) (2025)
- [TASTE: Task-Aware Out-of-Distribution Detection via Stein Operators](https://mlsanity.com/paper/2602.07640/) (2026)
- [Towards Robust Off-Policy Evaluation via Human Inputs](https://mlsanity.com/paper/2209.08682/) (2022)
- [Learning Run-time Safety Monitors for Machine Learning Components](https://mlsanity.com/paper/2406.16220/) (2024)
- [Spurious Correlations in Concept Drift: Can Explanatory Interaction Help?](https://mlsanity.com/paper/2407.16515/) (2024)
- [A generalized approach to label shift: the Conditional Probability Shift Model](https://mlsanity.com/paper/2503.02583/) (2025)
- [Explaining Concept Shift with Interpretable Feature Attribution](https://mlsanity.com/paper/2505.20634/) (2025)
- [Feature Shift Localization Network](https://mlsanity.com/paper/2506.09101/) (2025)
- [Deceptive Risk Minimization: Out-of-Distribution Generalization by Deceiving Distribution Shift Detectors](https://mlsanity.com/paper/2509.12081/) (2025)
- [Parameter-Efficient Temporal Domain Generalization](https://mlsanity.com/paper/2602.11965/) (2026)

## Abstract (arXiv)

While previous distribution shift detection approaches can identify if a shift has occurred, these approaches cannot localize which specific features have caused a distribution shift -- a critical step in diagnosing or fixing any underlying issue. For example, in military sensor networks, users will want to detect when one or more of the sensors has been compromised, and critically, they will want to know which specific sensors might be compromised. Thus, we first define a formalization of this problem as multiple conditional distribution hypothesis tests and propose both non-parametric and parametric statistical tests. For both efficiency and flexibility, we then propose to use a test statistic based on the density model score function (i.e. gradient with respect to the input) -- which can easily compute test statistics for all dimensions in a single forward and backward pass. Any density model could be used for computing the necessary statistics including deep density models such as normalizing flows or autoregressive models. We additionally develop methods for identifying when and where a shift occurs in multivariate time-series data and show results for multiple scenarios using realistic attack models on both simulated and real world data.