# Efficient Multi-Domain Network Learning by Covariance Normalization

arXiv: https://arxiv.org/abs/1906.10267 · HTML: https://mlsanity.com/paper/1906.10267/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jun 2019.

## Built on by (most cited first, 16 of 16)

- [KG-SP: Knowledge Guided Simple Primitives for Open World Compositional Zero-Shot Learning](https://mlsanity.com/paper/2205.06784/) (2022)
- [Deep vanishing point detection: Geometric priors make dataset variations vanish](https://mlsanity.com/paper/2203.08586/) (2022)
- [On the Effectiveness of LayerNorm Tuning for Continual Learning in Vision Transformers](https://mlsanity.com/paper/2308.09610/) (2023)
- [CDAD-Net: Bridging Domain Gaps in Generalized Category Discovery](https://mlsanity.com/paper/2404.05366/) (2024)
- [Multi-Domain Active Learning: Literature Review and Comparative Study](https://mlsanity.com/paper/2106.13516/) (2021)
- [Boosting Binary Masks for Multi-Domain Learning through Affine Transformations](https://mlsanity.com/paper/2103.13894/) (2021)
- [Disentangling Transfer and Interference in Multi-Domain Learning](https://mlsanity.com/paper/2107.05445/) (2021)
- [Reciprocal Normalization for Domain Adaptation](https://mlsanity.com/paper/2112.10474/) (2021)
- [Model-agnostic Multi-Domain Learning with Domain-Specific Adapters for Action Recognition](https://mlsanity.com/paper/2204.07270/) (2022)
- [An Empirical Study on Multi-Domain Robust Semantic Segmentation](https://mlsanity.com/paper/2212.04221/) (2022)
- [Multi-Domain Learning with Modulation Adapters](https://mlsanity.com/paper/2307.08528/) (2023)
- [Decoupled Training: Return of Frustratingly Easy Multi-Domain Learning](https://mlsanity.com/paper/2309.10302/) (2023)
- [Tilt and Average : Geometric Adjustment of the Last Layer for Recalibration](https://mlsanity.com/paper/2406.10017/) (2024)
- [Variational Learning of Disentangled Representations](https://mlsanity.com/paper/2506.17182/) (2025)
- [Separating Shared and Domain-Specific LoRAs for Multi-Domain Learning](https://mlsanity.com/paper/2508.02978/) (2025)
- [Robust Representation Learning for Unified Online Top-K Recommendation](https://mlsanity.com/paper/2310.15492/) (2023)

## Abstract (arXiv)

The problem of multi-domain learning of deep networks is considered. An adaptive layer is induced per target domain and a novel procedure, denoted covariance normalization (CovNorm), proposed to reduce its parameters. CovNorm is a data driven method of fairly simple implementation, requiring two principal component analyzes (PCA) and fine-tuning of a mini-adaptation layer. Nevertheless, it is shown, both theoretically and experimentally, to have several advantages over previous approaches, such as batch normalization or geometric matrix approximations. Furthermore, CovNorm can be deployed both when target datasets are available sequentially or simultaneously. Experiments show that, in both cases, it has performance comparable to a fully fine-tuned network, using as few as 0.13% of the corresponding parameters per target domain.