# Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning

arXiv: https://arxiv.org/abs/1702.08811 · HTML: https://mlsanity.com/paper/1702.08811/

## Key facts

- Cited by 192 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2017.

## Built on by (most cited first, 30 of 192)

- [The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalization](https://mlsanity.com/paper/2112.00463/) (2021)
- [Domain Adaptation via Prompt Learning](https://mlsanity.com/paper/2202.06687/) (2022)
- [Exact Feature Distribution Matching for Arbitrary Style Transfer and Domain Generalization](https://mlsanity.com/paper/2203.07740/) (2022)
- [Cycle Self-Training for Domain Adaptation](https://mlsanity.com/paper/2103.03571/) (2021)
- [Deep Subdomain Adaptation Network for Image Classification](https://mlsanity.com/paper/2106.09388/) (2021)
- [Align Your Prompts: Test-Time Prompting with Distribution Alignment for Zero-Shot Generalization](https://mlsanity.com/paper/2311.01459/) (2023)
- [Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectives](https://mlsanity.com/paper/2208.07422/) (2022)
- [Domain Impression: A Source Data Free Domain Adaptation Method](https://mlsanity.com/paper/2102.09003/) (2021)
- [Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training Data](https://mlsanity.com/paper/2108.01099/) (2021)
- [GFT: Graph Foundation Model with Transferable Tree Vocabulary](https://mlsanity.com/paper/2411.06070/) (2024)
- [Cross-domain Contrastive Learning for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2106.05528/) (2021)
- [DELTA: degradation-free fully test-time adaptation](https://mlsanity.com/paper/2301.13018/) (2023)
- [Cross-Domain Gradient Discrepancy Minimization for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2106.04151/) (2021)
- [Empowering Graph Representation Learning with Test-Time Graph Transformation](https://mlsanity.com/paper/2210.03561/) (2022)
- [Transferability in Deep Learning: A Survey](https://mlsanity.com/paper/2201.05867/) (2022)
- [MetaAlign: Coordinating Domain Alignment and Classification for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2103.13575/) (2021)
- [Patch-Mix Transformer for Unsupervised Domain Adaptation: A Game Perspective](https://mlsanity.com/paper/2303.13434/) (2023)
- [Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning](https://mlsanity.com/paper/2211.06612/) (2022)
- [Source-Free Adaptation to Measurement Shift via Bottom-Up Feature Restoration](https://mlsanity.com/paper/2107.05446/) (2021)
- [Style Normalization and Restitution for Domain Generalization and Adaptation](https://mlsanity.com/paper/2101.00588/) (2021)
- [Structural Re-weighting Improves Graph Domain Adaptation](https://mlsanity.com/paper/2306.03221/) (2023)
- [Revisiting Realistic Test-Time Training: Sequential Inference and Adaptation by Anchored Clustering](https://mlsanity.com/paper/2206.02721/) (2022)
- [ActMAD: Activation Matching to Align Distributions for Test-Time-Training](https://mlsanity.com/paper/2211.12870/) (2022)
- [Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspective](https://mlsanity.com/paper/2111.14820/) (2021)
- [Adversarial Feature Augmentation for Cross-domain Few-shot Classification](https://mlsanity.com/paper/2208.11021/) (2022)
- [Video Test-Time Adaptation for Action Recognition](https://mlsanity.com/paper/2211.15393/) (2022)
- [ProxyMix: Proxy-based Mixup Training with Label Refinery for Source-Free Domain Adaptation](https://mlsanity.com/paper/2205.14566/) (2022)
- [Towards Real-World Test-Time Adaptation: Tri-Net Self-Training with Balanced Normalization](https://mlsanity.com/paper/2309.14949/) (2023)
- [Model Adaptation: Unsupervised Domain Adaptation without Source Data](https://mlsanity.com/paper/2502.19316/) (2025)
- [ToAlign: Task-oriented Alignment for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2106.10812/) (2021)

## Abstract (arXiv)

The learning of domain-invariant representations in the context of domain adaptation with neural networks is considered. We propose a new regularization method that minimizes the discrepancy between domain-specific latent feature representations directly in the hidden activation space. Although some standard distribution matching approaches exist that can be interpreted as the matching of weighted sums of moments, e.g. Maximum Mean Discrepancy (MMD), an explicit order-wise matching of higher order moments has not been considered before. We propose to match the higher order central moments of probability distributions by means of order-wise moment differences. Our model does not require computationally expensive distance and kernel matrix computations. We utilize the equivalent representation of probability distributions by moment sequences to define a new distance function, called Central Moment Discrepancy (CMD). We prove that CMD is a metric on the set of probability distributions on a compact interval. We further prove that convergence of probability distributions on compact intervals w.r.t. the new metric implies convergence in distribution of the respective random variables. We test our approach on two different benchmark data sets for object recognition (Office) and sentiment analysis of product reviews (Amazon reviews). CMD achieves a new state-of-the-art performance on most domain adaptation tasks of Office and outperforms networks trained with MMD, Variational Fair Autoencoders and Domain Adversarial Neural Networks on Amazon reviews. In addition, a post-hoc parameter sensitivity analysis shows that the new approach is stable w.r.t. parameter changes in a certain interval. The source code of the experiments is publicly available.