# Model-agnostic Multi-Domain Learning with Domain-Specific Adapters for Action Recognition

arXiv: https://arxiv.org/abs/2204.07270 · HTML: https://mlsanity.com/paper/2204.07270/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 31 references that are in the map.
- First posted on arXiv 15 Apr 2022.

## Builds on (most cited first, 26 of 26)

- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [The "something something" video database for learning and evaluating visual common sense](https://mlsanity.com/paper/1706.04261/) (2017)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [X3D: Expanding Architectures for Efficient Video Recognition](https://mlsanity.com/paper/2004.04730/) (2020)
- [Deep Visual Domain Adaptation: A Survey](https://mlsanity.com/paper/1802.03601/) (2018)
- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [Learning multiple visual domains with residual adapters](https://mlsanity.com/paper/1705.08045/) (2017)
- [Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification](https://mlsanity.com/paper/1712.04851/) (2017)
- [Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks](https://mlsanity.com/paper/1711.10305/) (2017)
- [A Survey of Unsupervised Deep Domain Adaptation](https://mlsanity.com/paper/1812.02849/) (2018)
- [Video Action Transformer Network](https://mlsanity.com/paper/1812.02707/) (2018)
- [Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition](https://mlsanity.com/paper/1708.07632/) (2017)
- [Efficient parametrization of multi-domain deep neural networks](https://mlsanity.com/paper/1803.10082/) (2018)
- [Multi-Modal Domain Adaptation for Fine-Grained Action Recognition](https://mlsanity.com/paper/2001.09691/) (2020)
- [A survey on domain adaptation theory: learning bounds and theoretical guarantees](https://mlsanity.com/paper/2004.11829/) (2020)
- [Temporal Attentive Alignment for Large-Scale Video Domain Adaptation](https://mlsanity.com/paper/1907.12743/) (2019)
- [Adversarial Cross-Domain Action Recognition with Co-Attention](https://mlsanity.com/paper/1912.10405/) (2019)
- [PyTorchVideo: A Deep Learning Library for Video Understanding](https://mlsanity.com/paper/2111.09887/) (2021)
- [S3D: Single Shot multi-Span Detector via Fully 3D Convolutional Networks](https://mlsanity.com/paper/1807.08069/) (2018)
- [Efficient Multi-Domain Network Learning by Covariance Normalization](https://mlsanity.com/paper/1906.10267/) (2019)
- [Video Action Understanding](https://mlsanity.com/paper/2010.06647/) (2020)
- [The End-of-End-to-End: A Video Understanding Pentathlon Challenge (2020)](https://mlsanity.com/paper/2008.00744/) (2020)

## Built on by (most cited first, 3 of 3)

- [TransNet: A Transfer Learning-Based Network for Human Action Recognition](https://mlsanity.com/paper/2309.06951/) (2023)
- [Variational Learning of Disentangled Representations](https://mlsanity.com/paper/2506.17182/) (2025)
- [Joint learning of images and videos with a single Vision Transformer](https://mlsanity.com/paper/2308.10533/) (2023)

## Abstract (arXiv)

In this paper, we propose a multi-domain learning model for action recognition. The proposed method inserts domain-specific adapters between layers of domain-independent layers of a backbone network. Unlike a multi-head network that switches classification heads only, our model switches not only the heads, but also the adapters for facilitating to learn feature representations universal to multiple domains. Unlike prior works, the proposed method is model-agnostic and doesn't assume model structures unlike prior works. Experimental results on three popular action recognition datasets (HMDB51, UCF101, and Kinetics-400) demonstrate that the proposed method is more effective than a multi-head architecture and more efficient than separately training models for each domain.