# On Generalizing Beyond Domains in Cross-Domain Continual Learning

arXiv: https://arxiv.org/abs/2203.03970 · HTML: https://mlsanity.com/paper/2203.03970/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 45 references that are in the map.
- First posted on arXiv 8 Mar 2022.

## Builds on (most cited first, 28 of 28)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Domain Generalization with MixStyle](https://mlsanity.com/paper/2104.02008/) (2021)
- [Class-incremental learning: survey and performance evaluation on image classification](https://mlsanity.com/paper/2010.15277/) (2020)
- [Contrastive Adaptation Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1901.00976/) (2019)
- [Domain Generalization via Model-Agnostic Learning of Semantic Features](https://mlsanity.com/paper/1910.13580/) (2019)
- [Semi-supervised Domain Adaptation via Minimax Entropy](https://mlsanity.com/paper/1904.06487/) (2019)
- [Mnemonics Training: Multi-Class Incremental Learning without Forgetting](https://mlsanity.com/paper/2002.10211/) (2020)
- [Scatter Component Analysis: A Unified Framework for Domain Adaptation and Domain Generalization](https://mlsanity.com/paper/1510.04373/) (2015)
- [On Learning the Geodesic Path for Incremental Learning](https://mlsanity.com/paper/2104.08572/) (2021)
- [Continual Adaptation of Visual Representations via Domain Randomization and Meta-learning](https://mlsanity.com/paper/2012.04324/) (2020)
- [Class-Incremental Domain Adaptation](https://mlsanity.com/paper/2008.01389/) (2020)
- [Cross-Domain Similarity Learning for Face Recognition in Unseen Domains](https://mlsanity.com/paper/2103.07503/) (2021)
- [Domain-Invariant Projection Learning for Zero-Shot Recognition](https://mlsanity.com/paper/1810.08326/) (2018)

## Built on by (most cited first, 25 of 25)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learning](https://mlsanity.com/paper/2309.14062/) (2023)
- [Weighted Ensemble Models Are Strong Continual Learners](https://mlsanity.com/paper/2312.08977/) (2023)
- [Online DPO: Online Direct Preference Optimization with Fast-Slow Chasing](https://mlsanity.com/paper/2406.05534/) (2024)
- [Cloud-Device Collaborative Adaptation to Continual Changing Environments in the Real-world](https://mlsanity.com/paper/2212.00972/) (2022)
- [CBA: Improving Online Continual Learning via Continual Bias Adaptor](https://mlsanity.com/paper/2308.06925/) (2023)
- [Hyperparameters in Continual Learning: A Reality Check](https://mlsanity.com/paper/2403.09066/) (2024)
- [Learning with Style: Continual Semantic Segmentation Across Tasks and Domains](https://mlsanity.com/paper/2210.07016/) (2022)
- [Domain Generalization Guided by Gradient Signal to Noise Ratio of Parameters](https://mlsanity.com/paper/2310.07361/) (2023)
- [Versatile Incremental Learning: Towards Class and Domain-Agnostic Incremental Learning](https://mlsanity.com/paper/2409.10956/) (2024)
- [BECAME: BayEsian Continual Learning with Adaptive Model MErging](https://mlsanity.com/paper/2504.02666/) (2025)
- [Train with Perturbation, Infer after Merging: A Two-Stage Framework for Continual Learning](https://mlsanity.com/paper/2505.22389/) (2025)
- [Generalized Few-Shot Continual Learning with Contrastive Mixture of Adapters](https://mlsanity.com/paper/2302.05936/) (2023)
- [Domain-aware Triplet loss in Domain Generalization](https://mlsanity.com/paper/2303.01233/) (2023)
- [Multivariate Prototype Representation for Domain-Generalized Incremental Learning](https://mlsanity.com/paper/2309.13563/) (2023)
- [A Benchmark Grocery Dataset of Realworld Point Clouds From Single View](https://mlsanity.com/paper/2402.07819/) (2024)
- [Towards Cross-Domain Continual Learning](https://mlsanity.com/paper/2402.12490/) (2024)
- [Dual-CBA: Improving Online Continual Learning via Dual Continual Bias Adaptors from a Bi-level Optimization Perspective](https://mlsanity.com/paper/2408.13991/) (2024)
- [Cross-Sample Relational Fusion: Unifying Domain Generalization and Class-Incremental Learning](https://mlsanity.com/paper/2605.08839/) (2026)
- [Overcoming Data Inequality across Domains with Semi-Supervised Domain Generalization](https://mlsanity.com/paper/2403.05209/) (2024)
- [Dual-Alignment Knowledge Retention for Continual Medical Image Segmentation](https://mlsanity.com/paper/2507.03638/) (2025)
- [Adaptive Model Ensemble for Continual Learning](https://mlsanity.com/paper/2509.19819/) (2025)
- [Hierarchical Dataset Selection for High-Quality Data Sharing](https://mlsanity.com/paper/2512.10952/) (2025)
- [RoAD Benchmark: How LiDAR Models Fail under Coupled Domain Shifts and Label Evolution](https://mlsanity.com/paper/2601.07855/) (2026)
- [Understanding the Role of Rehearsal Scale in Continual Learning under Varying Model Capacities](https://mlsanity.com/paper/2602.20791/) (2026)

## Abstract (arXiv)

Humans have the ability to accumulate knowledge of new tasks in varying conditions, but deep neural networks often suffer from catastrophic forgetting of previously learned knowledge after learning a new task. Many recent methods focus on preventing catastrophic forgetting under the assumption of train and test data following similar distributions. In this work, we consider a more realistic scenario of continual learning under domain shifts where the model must generalize its inference to an unseen domain. To this end, we encourage learning semantically meaningful features by equipping the classifier with class similarity metrics as learning parameters which are obtained through Mahalanobis similarity computations. Learning of the backbone representation along with these extra parameters is done seamlessly in an end-to-end manner. In addition, we propose an approach based on the exponential moving average of the parameters for better knowledge distillation. We demonstrate that, to a great extent, existing continual learning algorithms fail to handle the forgetting issue under multiple distributions, while our proposed approach learns new tasks under domain shift with accuracy boosts up to 10% on challenging datasets such as DomainNet and OfficeHome.