# Maximum Mean Discrepancy for Generalization in the Presence of Distribution and Missingness Shift

arXiv: https://arxiv.org/abs/2111.10344 · HTML: https://mlsanity.com/paper/2111.10344/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 31 references that are in the map.
- First posted on arXiv 19 Nov 2021.

## Builds on (most cited first, 13 of 13)

- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Generative Moment Matching Networks](https://mlsanity.com/paper/1502.02761/) (2015)
- [MMD GAN: Towards Deeper Understanding of Moment Matching Network](https://mlsanity.com/paper/1705.08584/) (2017)
- [Training generative neural networks via Maximum Mean Discrepancy optimization](https://mlsanity.com/paper/1505.03906/) (2015)
- [Mind the Class Weight Bias: Weighted Maximum Mean Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1705.00609/) (2017)
- [Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy](https://mlsanity.com/paper/1611.04488/) (2016)
- [HoMM: Higher-order Moment Matching for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1912.11976/) (2019)
- [A Linear-Time Kernel Goodness-of-Fit Test](https://mlsanity.com/paper/1705.07673/) (2017)
- [Estimation of Corporate Greenhouse Gas Emissions via Machine Learning](https://mlsanity.com/paper/2109.04318/) (2021)

## Built on by (most cited first, 11 of 11)

- [PhaseFormer: From Patches to Phases for Efficient and Effective Time Series Forecasting](https://mlsanity.com/paper/2510.04134/) (2025)
- [Introducing Non-Linear Activations into Quantum Generative Models](https://mlsanity.com/paper/2205.14506/) (2022)
- [FRAMES-VQA: Benchmarking Fine-Tuning Robustness across Multi-Modal Shifts in Visual Question Answering](https://mlsanity.com/paper/2505.21755/) (2025)
- [MIRRAMS: Learning Robust Tabular Models under Unseen Missingness Shifts](https://mlsanity.com/paper/2507.08280/) (2025)
- [Attention Head Purification: A New Perspective to Harness CLIP for Domain Generalization](https://mlsanity.com/paper/2412.07226/) (2024)
- [Knowledge Rectification for Camouflaged Object Detection: Unlocking Insights from Low-Quality Data](https://mlsanity.com/paper/2503.22180/) (2025)
- [Diffusion-based Target Sampler for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2303.12724/) (2023)
- [Optimal Transport for Kernel Gaussian Mixture Models](https://mlsanity.com/paper/2310.18586/) (2023)
- [FedDA-TSformer: Federated Domain Adaptation with Vision TimeSformer for Left Ventricle Segmentation on Gated Myocardial Perfusion SPECT Image](https://mlsanity.com/paper/2502.16709/) (2025)
- [Towards a Proactive Autoscaling Framework for Data Stream Processing at the Edge using GRU and Transfer Learning](https://mlsanity.com/paper/2507.14597/) (2025)
- [UF-AMA: A unified framework for cross-domain emotion recognition via adaptive multimodal alignment](https://mlsanity.com/paper/2606.00170/) (2026)

## Abstract (arXiv)

Covariate shifts are a common problem in predictive modeling on real-world problems. This paper proposes addressing the covariate shift problem by minimizing Maximum Mean Discrepancy (MMD) statistics between the training and test sets in either feature input space, feature representation space, or both. We designed three techniques that we call MMD Representation, MMD Mask, and MMD Hybrid to deal with the scenarios where only a distribution shift exists, only a missingness shift exists, or both types of shift exist, respectively. We find that integrating an MMD loss component helps models use the best features for generalization and avoid dangerous extrapolation as much as possible for each test sample. Models treated with this MMD approach show better performance, calibration, and extrapolation on the test set.