# Multi-Domain Learning with Modulation Adapters

arXiv: https://arxiv.org/abs/2307.08528 · HTML: https://mlsanity.com/paper/2307.08528/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 39 references that are in the map.
- First posted on arXiv 17 Jul 2023.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [Learning multiple visual domains with residual adapters](https://mlsanity.com/paper/1705.08045/) (2017)
- [Weakly Supervised Deep Detection Networks](https://mlsanity.com/paper/1511.02853/) (2015)
- [Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Feature](https://mlsanity.com/paper/1505.00855/) (2015)
- [Efficient parametrization of multi-domain deep neural networks](https://mlsanity.com/paper/1803.10082/) (2018)
- [SpotTune: Transfer Learning through Adaptive Fine-tuning](https://mlsanity.com/paper/1811.08737/) (2018)
- [Incremental Learning Through Deep Adaptation](https://mlsanity.com/paper/1705.04228/) (2017)
- [Image Generators with Conditionally-Independent Pixel Synthesis](https://mlsanity.com/paper/2011.13775/) (2020)
- [Image Generation From Small Datasets via Batch Statistics Adaptation](https://mlsanity.com/paper/1904.01774/) (2019)
- [Few-Shot Adaptation of Generative Adversarial Networks](https://mlsanity.com/paper/2010.11943/) (2020)
- [Universal representations:The missing link between faces, text, planktons, and cat breeds](https://mlsanity.com/paper/1701.07275/) (2017)
- [Depthwise Convolution is All You Need for Learning Multiple Visual Domains](https://mlsanity.com/paper/1902.00927/) (2019)
- [Adding New Tasks to a Single Network with Weight Transformations using Binary Masks](https://mlsanity.com/paper/1805.11119/) (2018)
- [Budget-Aware Adapters for Multi-Domain Learning](https://mlsanity.com/paper/1905.06242/) (2019)
- [Efficient Multi-Domain Network Learning by Covariance Normalization](https://mlsanity.com/paper/1906.10267/) (2019)
- [Not all domains are equally complex: Adaptive Multi-Domain Learning](https://mlsanity.com/paper/2003.11504/) (2020)

## Built on by (most cited first, 1 of 1)

- [Budget-Aware Pruning: Handling Multiple Domains with Less Parameters](https://mlsanity.com/paper/2309.11464/) (2023)

## Abstract (arXiv)

Deep convolutional networks are ubiquitous in computer vision, due to their excellent performance across different tasks for various domains. Models are, however, often trained in isolation for each task, failing to exploit relatedness between tasks and domains to learn more compact models that generalise better in low-data regimes. Multi-domain learning aims to handle related tasks, such as image classification across multiple domains, simultaneously. Previous work on this problem explored the use of a pre-trained and fixed domain-agnostic base network, in combination with smaller learnable domain-specific adaptation modules. In this paper, we introduce Modulation Adapters, which update the convolutional filter weights of the model in a multiplicative manner for each task. Parameterising these adaptation weights in a factored manner allows us to scale the number of per-task parameters in a flexible manner, and to strike different parameter-accuracy trade-offs. We evaluate our approach on the Visual Decathlon challenge, composed of ten image classification tasks across different domains, and on the ImageNet-to-Sketch benchmark, which consists of six image classification tasks. Our approach yields excellent results, with accuracies that are comparable to or better than those of existing state-of-the-art approaches.