# Causal Covariate Shift Correction using Fisher information penalty

arXiv: https://arxiv.org/abs/2502.15756 · HTML: https://mlsanity.com/paper/2502.15756/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 26 references that are in the map.
- First posted on arXiv 11 Feb 2025.

## Builds on (most cited first, 9 of 9)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Deep Learning for Classical Japanese Literature](https://mlsanity.com/paper/1812.01718/) (2018)
- [Revisiting Natural Gradient for Deep Networks](https://mlsanity.com/paper/1301.3584/) (2013)
- [MNIST-C: A Robustness Benchmark for Computer Vision](https://mlsanity.com/paper/1906.02337/) (2019)
- [Rethinking Importance Weighting for Deep Learning under Distribution Shift](https://mlsanity.com/paper/2006.04662/) (2020)
- [A New Lower Bound for Kullback-Leibler Divergence Based on Hammersley-Chapman-Robbins Bound](https://mlsanity.com/paper/1907.00288/) (2019)

## Built on by (most cited first, 5 of 5)

- [Calibrated and Robust Foundation Models for Vision-Language and Medical Image Tasks Under Distribution Shift](https://mlsanity.com/paper/2507.09222/) (2025)
- [CalTwin: Towards Calibrated, Shift-Robust Medical World Models via Fisher-Information Regularisation](https://mlsanity.com/paper/2607.26752/) (2026)
- [Efficient Learning Under Density Shift in Incremental Settings Using Cram'er-Rao-Based Regularization](https://mlsanity.com/paper/2502.12949/) (2025)
- [Adapting to Fragmented and Evolving Data: A Fisher Information Perspective](https://mlsanity.com/paper/2507.18996/) (2025)
- [Technical note on Fisher Information for Robust Federated Cross-Validation](https://mlsanity.com/paper/2510.03838/) (2025)

## Abstract (arXiv)

Evolving feature densities across batches of training data bias cross-validation, making model selection and assessment unreliable (citesugiyama2012machine). This work takes a distributed density estimation angle to the training setting where data are temporally distributed. textitCausal Covariate Shift Correction (C3), accumulates knowledge about the data density of a training batch using Fisher Information, and using it to penalize the loss in all subsequent batches. The penalty improves accuracy by 12.9% over the full-dataset baseline, by 20.3% accuracy at maximum in batchwise and 5.9% at minimum in foldwise benchmarks.