# Learning Cross-modal Contrastive Features for Video Domain Adaptation

arXiv: https://arxiv.org/abs/2108.11974 · HTML: https://mlsanity.com/paper/2108.11974/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 68 references that are in the map.
- First posted on arXiv 26 Aug 2021.

## Builds on (most cited first, 30 of 46)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1811.12833/) (2018)
- [Look, Listen and Learn](https://mlsanity.com/paper/1705.08168/) (2017)
- [Domain Adaptive Faster R-CNN for Object Detection in the Wild](https://mlsanity.com/paper/1803.03243/) (2018)
- [Unsupervised Learning of Video Representations using LSTMs](https://mlsanity.com/paper/1502.04681/) (2015)
- [Confidence Regularized Self-Training](https://mlsanity.com/paper/1908.09822/) (2019)
- [Hard Negative Mixing for Contrastive Learning](https://mlsanity.com/paper/2010.01028/) (2020)
- [Temporal Relational Reasoning in Videos](https://mlsanity.com/paper/1711.08496/) (2017)
- [Bidirectional Learning for Domain Adaptation of Semantic Segmentation](https://mlsanity.com/paper/1904.10620/) (2019)
- [Strong-Weak Distribution Alignment for Adaptive Object Detection](https://mlsanity.com/paper/1812.04798/) (2018)
- [Self-Supervised Learning by Cross-Modal Audio-Video Clustering](https://mlsanity.com/paper/1911.12667/) (2019)
- [Cooperative Learning of Audio and Video Models from Self-Supervised Synchronization](https://mlsanity.com/paper/1807.00230/) (2018)
- [Self-supervised Co-training for Video Representation Learning](https://mlsanity.com/paper/2010.09709/) (2020)
- [Unsupervised Representation Learning by Sorting Sequences](https://mlsanity.com/paper/1708.01246/) (2017)
- [Domain Adaptation for Structured Output via Discriminative Patch Representations](https://mlsanity.com/paper/1901.05427/) (2019)

## Built on by (most cited first, 30 of 47)

- [EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real World](https://mlsanity.com/paper/2403.16182/) (2024)
- [An Outlook into the Future of Egocentric Vision](https://mlsanity.com/paper/2308.07123/) (2023)
- [MM-TTA: Multi-Modal Test-Time Adaptation for 3D Semantic Segmentation](https://mlsanity.com/paper/2204.12667/) (2022)
- [Domain Adaptation for Time Series Under Feature and Label Shifts](https://mlsanity.com/paper/2302.03133/) (2023)
- [E2(GO)MOTION: Motion Augmented Event Stream for Egocentric Action Recognition](https://mlsanity.com/paper/2112.03596/) (2021)
- [SimMMDG: A Simple and Effective Framework for Multi-modal Domain Generalization](https://mlsanity.com/paper/2310.19795/) (2023)
- [Multi-Modal Cross-Domain Alignment Network for Video Moment Retrieval](https://mlsanity.com/paper/2209.11572/) (2022)
- [Audio-Adaptive Activity Recognition Across Video Domains](https://mlsanity.com/paper/2203.14240/) (2022)
- [What can a cook in Italy teach a mechanic in India? Action Recognition Generalisation Over Scenarios and Locations](https://mlsanity.com/paper/2306.08713/) (2023)
- [Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Models](https://mlsanity.com/paper/2501.18592/) (2025)
- [Unsupervised Video Domain Adaptation for Action Recognition: A Disentanglement Perspective](https://mlsanity.com/paper/2208.07365/) (2022)
- [Balance Act: Mitigating Hubness in Cross-Modal Retrieval with Query and Gallery Banks](https://mlsanity.com/paper/2310.11612/) (2023)
- [Diversifying Spatial-Temporal Perception for Video Domain Generalization](https://mlsanity.com/paper/2310.17942/) (2023)
- [AutoLabel: CLIP-based framework for Open-set Video Domain Adaptation](https://mlsanity.com/paper/2304.01110/) (2023)
- [Synchronization is All You Need: Exocentric-to-Egocentric Transfer for Temporal Action Segmentation with Unlabeled Synchronized Video Pairs](https://mlsanity.com/paper/2312.02638/) (2023)
- [Towards Multimodal Open-Set Domain Generalization and Adaptation through Self-supervision](https://mlsanity.com/paper/2407.01518/) (2024)
- [XOV-Action: Towards Generalizable Open-Vocabulary Action Recognition](https://mlsanity.com/paper/2403.01560/) (2024)
- [Video Unsupervised Domain Adaptation with Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/2211.10412/) (2022)
- [Towards Robust Multimodal Open-set Test-time Adaptation via Adaptive Entropy-aware Optimization](https://mlsanity.com/paper/2501.13924/) (2025)
- [Exo2EgoDVC: Dense Video Captioning of Egocentric Procedural Activities Using Web Instructional Videos](https://mlsanity.com/paper/2311.16444/) (2023)
- [Unsupervised Domain Adaptation for Video Transformers in Action Recognition](https://mlsanity.com/paper/2207.12842/) (2022)
- [Spatio-Temporal Pixel-Level Contrastive Learning-based Source-Free Domain Adaptation for Video Semantic Segmentation](https://mlsanity.com/paper/2303.14361/) (2023)
- [EMAG: Ego-motion Aware and Generalizable 2D Hand Forecasting from Egocentric Videos](https://mlsanity.com/paper/2405.20030/) (2024)
- [Human-Centric Transformer for Domain Adaptive Action Recognition](https://mlsanity.com/paper/2407.10860/) (2024)
- [Leveraging Endo- and Exo-Temporal Regularization for Black-box Video Domain Adaptation](https://mlsanity.com/paper/2208.05187/) (2022)
- [SPCL: A New Framework for Domain Adaptive Semantic Segmentation via Semantic Prototype-based Contrastive Learning](https://mlsanity.com/paper/2111.12358/) (2021)
- [CycDA: Unsupervised Cycle Domain Adaptation from Image to Video](https://mlsanity.com/paper/2203.16244/) (2022)
- [Source-free Video Domain Adaptation by Learning from Noisy Labels](https://mlsanity.com/paper/2311.18572/) (2023)
- [Learning Contrastive Multimodal Fusion with Improved Modality Dropout for Disease Detection and Prediction](https://mlsanity.com/paper/2509.18284/) (2025)
- [Unsupervised Modality Adaptation with Text-to-Image Diffusion Models for Semantic Segmentation](https://mlsanity.com/paper/2410.21708/) (2024)

## Abstract (arXiv)

Learning transferable and domain adaptive feature representations from videos is important for video-relevant tasks such as action recognition. Existing video domain adaptation methods mainly rely on adversarial feature alignment, which has been derived from the RGB image space. However, video data is usually associated with multi-modal information, e.g., RGB and optical flow, and thus it remains a challenge to design a better method that considers the cross-modal inputs under the cross-domain adaptation setting. To this end, we propose a unified framework for video domain adaptation, which simultaneously regularizes cross-modal and cross-domain feature representations. Specifically, we treat each modality in a domain as a view and leverage the contrastive learning technique with properly designed sampling strategies. As a result, our objectives regularize feature spaces, which originally lack the connection across modalities or have less alignment across domains. We conduct experiments on domain adaptive action recognition benchmark datasets, i.e., UCF, HMDB, and EPIC-Kitchens, and demonstrate the effectiveness of our components against state-of-the-art algorithms.