# Domain Invariant Representation Learning with Domain Density Transformations

arXiv: https://arxiv.org/abs/2102.05082 · HTML: https://mlsanity.com/paper/2102.05082/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 68 references that are in the map.
- First posted on arXiv 9 Feb 2021.

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [Domain Generalization by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1903.06864/) (2019)
- [Bridging Theory and Algorithm for Domain Adaptation](https://mlsanity.com/paper/1904.05801/) (2019)
- [Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks](https://mlsanity.com/paper/1612.05424/) (2016)
- [NIPS 2016 Tutorial: Generative Adversarial Networks](https://mlsanity.com/paper/1701.00160/) (2016)
- [Generalizing Across Domains via Cross-Gradient Training](https://mlsanity.com/paper/1804.10745/) (2018)
- [Domain Generalization for Object Recognition with Multi-task Autoencoders](https://mlsanity.com/paper/1508.07680/) (2015)
- [Coupled Generative Adversarial Networks](https://mlsanity.com/paper/1606.07536/) (2016)
- [Learning to Optimize Domain Specific Normalization for Domain Generalization](https://mlsanity.com/paper/1907.04275/) (2019)
- [Domain Generalization via Conditional Invariant Representation](https://mlsanity.com/paper/1807.08479/) (2018)
- [Efficient Domain Generalization via Common-Specific Low-Rank Decomposition](https://mlsanity.com/paper/2003.12815/) (2020)
- [Learning to Balance Specificity and Invariance for In and Out of Domain Generalization](https://mlsanity.com/paper/2008.12839/) (2020)
- [Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift](https://mlsanity.com/paper/2003.04475/) (2020)
- [Learning Robust Representations by Projecting Superficial Statistics Out](https://mlsanity.com/paper/1903.06256/) (2019)
- [DIVA: Domain Invariant Variational Autoencoders](https://mlsanity.com/paper/1905.10427/) (2019)
- [Learning to Learn with Variational Information Bottleneck for Domain Generalization](https://mlsanity.com/paper/2007.07645/) (2020)

## Built on by (most cited first, 30 of 53)

- [OoD-Bench: Quantifying and Understanding Two Dimensions of Out-of-Distribution Generalization](https://mlsanity.com/paper/2106.03721/) (2021)
- [Less Data, More Knowledge: Building Next Generation Semantic Communication Networks](https://mlsanity.com/paper/2211.14343/) (2022)
- [A Sentence Speaks a Thousand Images: Domain Generalization through Distilling CLIP with Language Guidance](https://mlsanity.com/paper/2309.12530/) (2023)
- [KL Guided Domain Adaptation](https://mlsanity.com/paper/2106.07780/) (2021)
- [DiPrompT: Disentangled Prompt Tuning for Multiple Latent Domain Generalization in Federated Learning](https://mlsanity.com/paper/2403.08506/) (2024)
- [Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectives](https://mlsanity.com/paper/2307.16851/) (2023)
- [Fairness and Accuracy under Domain Generalization](https://mlsanity.com/paper/2301.13323/) (2023)
- [The Two Dimensions of Worst-case Training and the Integrated Effect for Out-of-domain Generalization](https://mlsanity.com/paper/2204.04384/) (2022)
- [Generalizing to Evolving Domains with Latent Structure-Aware Sequential Autoencoder](https://mlsanity.com/paper/2205.07649/) (2022)
- [CIRCLe: Color Invariant Representation Learning for Unbiased Classification of Skin Lesions](https://mlsanity.com/paper/2208.13528/) (2022)
- [Beyond Pixels: Text Enhances Generalization in Real-World Image Restoration](https://mlsanity.com/paper/2412.00878/) (2024)
- [Energy-Based Test Sample Adaptation for Domain Generalization](https://mlsanity.com/paper/2302.11215/) (2023)
- [Generalizing across Temporal Domains with Koopman Operators](https://mlsanity.com/paper/2402.07834/) (2024)
- [Foresee What You Will Learn: Data Augmentation for Domain Generalization in Non-stationary Environment](https://mlsanity.com/paper/2301.07845/) (2023)
- [Moderately Distributional Exploration for Domain Generalization](https://mlsanity.com/paper/2304.13976/) (2023)
- [Invariant Anomaly Detection under Distribution Shifts: A Causal Perspective](https://mlsanity.com/paper/2312.14329/) (2023)
- [Beyond the Federation: Topology-aware Federated Learning for Generalization to Unseen Clients](https://mlsanity.com/paper/2407.04949/) (2024)
- [Generalizability of Adversarial Robustness Under Distribution Shifts](https://mlsanity.com/paper/2209.15042/) (2022)
- [Domain Adaptation with Cauchy-Schwarz Divergence](https://mlsanity.com/paper/2405.19978/) (2024)
- [Fundamental Limits of Perfect Concept Erasure](https://mlsanity.com/paper/2503.20098/) (2025)
- [FAMLP: A Frequency-Aware MLP-Like Architecture For Domain Generalization](https://mlsanity.com/paper/2203.12893/) (2022)
- [Robust Self-Supervised Learning with Lie Groups](https://mlsanity.com/paper/2210.13356/) (2022)
- [Non-stationary Domain Generalization: Theory and Algorithm](https://mlsanity.com/paper/2405.06816/) (2024)
- [From Invariant Representations to Invariant Data: Provable Robustness to Spurious Correlations via Noisy Counterfactual Matching](https://mlsanity.com/paper/2505.24843/) (2025)
- [Domain Decorrelation with Potential Energy Ranking](https://mlsanity.com/paper/2207.12194/) (2022)
- [Joint covariate-alignment and concept-alignment: a framework for domain generalization](https://mlsanity.com/paper/2208.00898/) (2022)
- [Geometric Understanding of Discriminability and Transferability for Visual Domain Adaptation](https://mlsanity.com/paper/2407.09524/) (2024)
- [Single-Domain Generalized Object Detection by Balancing Domain Diversity and Invariance](https://mlsanity.com/paper/2502.03835/) (2025)
- [Why Domain Generalization Fail? A View of Necessity and Sufficiency](https://mlsanity.com/paper/2502.10716/) (2025)
- [Bridging Distribution Shift and AI Safety: Conceptual and Methodological Synergies](https://mlsanity.com/paper/2505.22829/) (2025)

## Abstract (arXiv)

Domain generalization refers to the problem where we aim to train a model on data from a set of source domains so that the model can generalize to unseen target domains. Naively training a model on the aggregate set of data (pooled from all source domains) has been shown to perform suboptimally, since the information learned by that model might be domain-specific and generalize imperfectly to target domains. To tackle this problem, a predominant approach is to find and learn some domain-invariant information in order to use it for the prediction task. In this paper, we propose a theoretically grounded method to learn a domain-invariant representation by enforcing the representation network to be invariant under all transformation functions among domains. We also show how to use generative adversarial networks to learn such domain transformations to implement our method in practice. We demonstrate the effectiveness of our method on several widely used datasets for the domain generalization problem, on all of which we achieve competitive results with state-of-the-art models.