# Depthwise Convolution is All You Need for Learning Multiple Visual Domains

arXiv: https://arxiv.org/abs/1902.00927 · HTML: https://mlsanity.com/paper/1902.00927/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Feb 2019.

## Built on by (most cited first, 30 of 36)

- [3D UX-Net: A Large Kernel Volumetric ConvNet Modernizing Hierarchical Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2209.15076/) (2022)
- [Conv-Adapter: Exploring Parameter Efficient Transfer Learning for ConvNets](https://mlsanity.com/paper/2208.07463/) (2022)
- [MambaMixer: Efficient Selective State Space Models with Dual Token and Channel Selection](https://mlsanity.com/paper/2403.19888/) (2024)
- [Multi-domain Learning for Updating Face Anti-spoofing Models](https://mlsanity.com/paper/2208.11148/) (2022)
- [INTERN: A New Learning Paradigm Towards General Vision](https://mlsanity.com/paper/2111.08687/) (2021)
- [Learning Multiple Stock Trading Patterns with Temporal Routing Adaptor and Optimal Transport](https://mlsanity.com/paper/2106.12950/) (2021)
- [Tiny and Efficient Model for the Edge Detection Generalization](https://mlsanity.com/paper/2308.06468/) (2023)
- [Multi-Domain Incremental Learning for Semantic Segmentation](https://mlsanity.com/paper/2110.12205/) (2021)
- [Adapters Strike Back](https://mlsanity.com/paper/2406.06820/) (2024)
- [Benchmarking of DL Libraries and Models on Mobile Devices](https://mlsanity.com/paper/2202.06512/) (2022)
- [Fast Object Placement Assessment](https://mlsanity.com/paper/2205.14280/) (2022)
- [ESDMR-Net: A Lightweight Network With Expand-Squeeze and Dual Multiscale Residual Connections for Medical Image Segmentation](https://mlsanity.com/paper/2312.10585/) (2023)
- [D-Net: Dynamic Large Kernel with Dynamic Feature Fusion for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2403.10674/) (2024)
- [X-Learner: Learning Cross Sources and Tasks for Universal Visual Representation](https://mlsanity.com/paper/2203.08764/) (2022)
- [A Study on the Generality of Neural Network Structures for Monocular Depth Estimation](https://mlsanity.com/paper/2301.03169/) (2023)
- [SpotPatch: Parameter-Efficient Transfer Learning for Mobile Object Detection](https://mlsanity.com/paper/2101.01260/) (2021)
- [Disentangling Transfer and Interference in Multi-Domain Learning](https://mlsanity.com/paper/2107.05445/) (2021)
- [Improving Robustness Against Adversarial Attacks with Deeply Quantized Neural Networks](https://mlsanity.com/paper/2304.12829/) (2023)
- [Efficient CNN-based Super Resolution Algorithms for mmWave Mobile Radar Imaging](https://mlsanity.com/paper/2305.02092/) (2023)
- [AttCDCNet: Attention-enhanced Chest Disease Classification using X-Ray Images](https://mlsanity.com/paper/2410.15437/) (2024)
- [Multi-Domain Learning with Modulation Adapters](https://mlsanity.com/paper/2307.08528/) (2023)
- [Agricultural Object Detection with You Look Only Once (YOLO) Algorithm: A Bibliometric and Systematic Literature Review](https://mlsanity.com/paper/2401.10379/) (2024)
- [YOLO-Drone: An Efficient Object Detection Approach Using the GhostHead Network for Drone Images](https://mlsanity.com/paper/2511.10905/) (2025)
- [Convolutional Normalization](https://mlsanity.com/paper/2102.09685/) (2021)
- [End-to-end Keyword Spotting using Xception-1d](https://mlsanity.com/paper/2110.07498/) (2021)
- [Memory Efficient Adaptive Attention For Multiple Domain Learning](https://mlsanity.com/paper/2110.10969/) (2021)
- [EffCNet: An Efficient CondenseNet for Image Classification on NXP BlueBox](https://mlsanity.com/paper/2111.14243/) (2021)
- [Visualizing CoAtNet Predictions for Aiding Melanoma Detection](https://mlsanity.com/paper/2205.10515/) (2022)
- [Multi-domain learning CNN model for microscopy image classification](https://mlsanity.com/paper/2304.10616/) (2023)
- [A Semi-Self-Supervised Approach for Dense-Pattern Video Object Segmentation](https://mlsanity.com/paper/2406.05131/) (2024)

## Abstract (arXiv)

There is a growing interest in designing models that can deal with images from different visual domains. If there exists a universal structure in different visual domains that can be captured via a common parameterization, then we can use a single model for all domains rather than one model per domain. A model aware of the relationships between different domains can also be trained to work on new domains with less resources. However, to identify the reusable structure in a model is not easy. In this paper, we propose a multi-domain learning architecture based on depthwise separable convolution. The proposed approach is based on the assumption that images from different domains share cross-channel correlations but have domain-specific spatial correlations. The proposed model is compact and has minimal overhead when being applied to new domains. Additionally, we introduce a gating mechanism to promote soft sharing between different domains. We evaluate our approach on Visual Decathlon Challenge, a benchmark for testing the ability of multi-domain models. The experiments show that our approach can achieve the highest score while only requiring 50% of the parameters compared with the state-of-the-art approaches.