# Unsupervised Video Domain Adaptation for Action Recognition: A Disentanglement Perspective

arXiv: https://arxiv.org/abs/2208.07365 · HTML: https://mlsanity.com/paper/2208.07365/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 54 references that are in the map.
- First posted on arXiv 15 Aug 2022.

## Builds on (most cited first, 30 of 31)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [A Survey of Unsupervised Deep Domain Adaptation](https://mlsanity.com/paper/1812.02849/) (2018)
- [Temporal Relational Reasoning in Videos](https://mlsanity.com/paper/1711.08496/) (2017)
- [Exploring Object Relation in Mean Teacher for Cross-Domain Detection](https://mlsanity.com/paper/1904.11245/) (2019)
- [Disentangled Sequential Autoencoder](https://mlsanity.com/paper/1803.02991/) (2018)
- [DIVA: Domain Invariant Variational Autoencoders](https://mlsanity.com/paper/1905.10427/) (2019)
- [Multi-Modal Domain Adaptation for Fine-Grained Action Recognition](https://mlsanity.com/paper/2001.09691/) (2020)
- [Temporal Attentive Alignment for Large-Scale Video Domain Adaptation](https://mlsanity.com/paper/1907.12743/) (2019)
- [S3VAE: Self-Supervised Sequential VAE for Representation Disentanglement and Data Generation](https://mlsanity.com/paper/2005.11437/) (2020)
- [Uncertainty-Aware Unsupervised Domain Adaptation in Object Detection](https://mlsanity.com/paper/2103.00236/) (2021)
- [Learning Disentangled Semantic Representation for Domain Adaptation](https://mlsanity.com/paper/2012.11807/) (2020)
- [Dynamic Weighted Learning for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2103.13814/) (2021)
- [Adversarial Cross-Domain Action Recognition with Co-Attention](https://mlsanity.com/paper/1912.10405/) (2019)
- [Contrast and Mix: Temporal Contrastive Video Domain Adaptation with Background Mixing](https://mlsanity.com/paper/2110.15128/) (2021)
- [Learning Cross-modal Contrastive Features for Video Domain Adaptation](https://mlsanity.com/paper/2108.11974/) (2021)
- [ConDA: Unsupervised Domain Adaptation for LiDAR Segmentation via Regularized Domain Concatenation](https://mlsanity.com/paper/2111.15242/) (2021)
- [Spectral Unsupervised Domain Adaptation for Visual Recognition](https://mlsanity.com/paper/2106.06112/) (2021)
- [Graph Domain Adaptation: A Generative View](https://mlsanity.com/paper/2106.07482/) (2021)
- [Audio-Adaptive Activity Recognition Across Video Domains](https://mlsanity.com/paper/2203.14240/) (2022)
- [Contrastively Disentangled Sequential Variational Autoencoder](https://mlsanity.com/paper/2110.12091/) (2021)
- [Adversarial Bipartite Graph Learning for Video Domain Adaptation](https://mlsanity.com/paper/2007.15829/) (2020)
- [Multi-Head Distillation for Continual Unsupervised Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/2204.11667/) (2022)

## Built on by (most cited first, 21 of 21)

- [EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real World](https://mlsanity.com/paper/2403.16182/) (2024)
- [Robust Disentangled Counterfactual Learning for Physical Audiovisual Commonsense Reasoning](https://mlsanity.com/paper/2502.12425/) (2025)
- [We're Not Using Videos Effectively: An Updated Domain Adaptive Video Segmentation Baseline](https://mlsanity.com/paper/2402.00868/) (2024)
- [Tell, Don't Show!: Language Guidance Eases Transfer Across Domains in Images and Videos](https://mlsanity.com/paper/2403.05535/) (2024)
- [Temporal Source Recovery for Time-Series Source-Free Unsupervised Domain Adaptation](https://mlsanity.com/paper/2409.19635/) (2024)
- [CHASE: Learning Convex Hull Adaptive Shift for Skeleton-based Multi-Entity Action Recognition](https://mlsanity.com/paper/2410.07153/) (2024)
- [Exploring Few-Shot Adaptation for Activity Recognition on Diverse Domains](https://mlsanity.com/paper/2305.08420/) (2023)
- [SADA: Semantic adversarial unsupervised domain adaptation for Temporal Action Localization](https://mlsanity.com/paper/2312.13377/) (2023)
- [Transferable-guided Attention Is All You Need for Video Domain Adaptation](https://mlsanity.com/paper/2407.01375/) (2024)
- [An Evaluation of Large Pre-Trained Models for Gesture Recognition using Synthetic Videos](https://mlsanity.com/paper/2410.02152/) (2024)
- [Time Series Domain Adaptation via Latent Invariant Causal Mechanism](https://mlsanity.com/paper/2502.16637/) (2025)
- [Adversarially Masked Video Consistency for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2403.16242/) (2024)
- [Semantically Guided Action Anticipation](https://mlsanity.com/paper/2411.15557/) (2024)
- [DREAM: Domain-agnostic Reverse Engineering Attributes of Black-box Model](https://mlsanity.com/paper/2412.05842/) (2024)
- [Probabilistic Temporal Masked Attention for Cross-view Online Action Detection](https://mlsanity.com/paper/2508.17025/) (2025)
- [Modality-Collaborative Low-Rank Decomposers for Few-Shot Video Domain Adaptation](https://mlsanity.com/paper/2511.18711/) (2025)
- [Logi-PAR: Logic-Infused Patient Activity Recognition via Differentiable Rule](https://mlsanity.com/paper/2603.05184/) (2026)
- [From Virtual Environments to Real-World Trials: Emerging Trends in Autonomous Driving](https://mlsanity.com/paper/2603.17714/) (2026)
- [Autonomous Drift Learning in Data Streams: A Unified Perspective](https://mlsanity.com/paper/2605.01295/) (2026)
- [Return of Frustratingly Easy Unsupervised Video Domain Adaptation](https://mlsanity.com/paper/2605.19510/) (2026)
- [Multi-Scale Temporal Domain Alignment for Federated Video Domain Adaptation](https://mlsanity.com/paper/2608.29186/) (2026)

## Abstract (arXiv)

Unsupervised video domain adaptation is a practical yet challenging task. In this work, for the first time, we tackle it from a disentanglement view. Our key idea is to handle the spatial and temporal domain divergence separately through disentanglement. Specifically, we consider the generation of cross-domain videos from two sets of latent factors, one encoding the static information and another encoding the dynamic information. A Transfer Sequential VAE (TranSVAE) framework is then developed to model such generation. To better serve for adaptation, we propose several objectives to constrain the latent factors. With these constraints, the spatial divergence can be readily removed by disentangling the static domain-specific information out, and the temporal divergence is further reduced from both frame- and video-levels through adversarial learning. Extensive experiments on the UCF-HMDB, Jester, and Epic-Kitchens datasets verify the effectiveness and superiority of TranSVAE compared with several state-of-the-art approaches. Code is publicly available.