# Calibrating Class Weights with Multi-Modal Information for Partial Video Domain Adaptation

arXiv: https://arxiv.org/abs/2204.06187 · HTML: https://mlsanity.com/paper/2204.06187/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 56 references that are in the map.
- First posted on arXiv 13 Apr 2022.

## Builds on (most cited first, 28 of 28)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [Bridging Theory and Algorithm for Domain Adaptation](https://mlsanity.com/paper/1904.05801/) (2019)
- [Temporal Relational Reasoning in Videos](https://mlsanity.com/paper/1711.08496/) (2017)
- [Learning to Estimate Hidden Motions with Global Motion Aggregation](https://mlsanity.com/paper/2104.02409/) (2021)
- [A DIRT-T Approach to Unsupervised Domain Adaptation](https://mlsanity.com/paper/1802.08735/) (2018)
- [Partial Adversarial Domain Adaptation](https://mlsanity.com/paper/1808.04205/) (2018)
- [Multi-Modal Domain Adaptation for Fine-Grained Action Recognition](https://mlsanity.com/paper/2001.09691/) (2020)
- [Temporal Attentive Alignment for Large-Scale Video Domain Adaptation](https://mlsanity.com/paper/1907.12743/) (2019)
- [Importance Weighted Adversarial Nets for Partial Domain Adaptation](https://mlsanity.com/paper/1803.09210/) (2018)
- [Cluster Alignment with a Teacher for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1903.09980/) (2019)
- [Learning to Transfer Examples for Partial Domain Adaptation](https://mlsanity.com/paper/1903.12230/) (2019)
- [Partial Transfer Learning with Selective Adversarial Networks](https://mlsanity.com/paper/1707.07901/) (2017)
- [Adversarial Cross-Domain Action Recognition with Co-Attention](https://mlsanity.com/paper/1912.10405/) (2019)
- [ARID: A New Dataset for Recognizing Action in the Dark](https://mlsanity.com/paper/2006.03876/) (2020)
- [Aligning Correlation Information for Domain Adaptation in Action Recognition](https://mlsanity.com/paper/2107.04932/) (2021)
- [Multi-Source Video Domain Adaptation with Temporal Attentive Moment Alignment](https://mlsanity.com/paper/2109.09964/) (2021)
- [Partial Video Domain Adaptation with Partial Adversarial Temporal Attentive Network](https://mlsanity.com/paper/2107.04941/) (2021)
- [Adaptively-Accumulated Knowledge Transfer for Partial Domain Adaptation](https://mlsanity.com/paper/2008.11873/) (2020)
- [Learning Transferable Self-attentive Representations for Action Recognition in Untrimmed Videos with Weak Supervision](https://mlsanity.com/paper/1902.07370/) (2019)

## Built on by (most cited first, 4 of 4)

- [SWL-Adapt: An Unsupervised Domain Adaptation Model with Sample Weight Learning for Cross-User Wearable Human Activity Recognition](https://mlsanity.com/paper/2212.00724/) (2022)
- [Can We Evaluate Domain Adaptation Models Without Target-Domain Labels?](https://mlsanity.com/paper/2305.18712/) (2023)
- [Video Unsupervised Domain Adaptation with Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/2211.10412/) (2022)
- [Augmenting and Aligning Snippets for Few-Shot Video Domain Adaptation](https://mlsanity.com/paper/2303.10451/) (2023)

## Abstract (arXiv)

Assuming the source label space subsumes the target one, Partial Video Domain Adaptation (PVDA) is a more general and practical scenario for cross-domain video classification problems. The key challenge of PVDA is to mitigate the negative transfer caused by the source-only outlier classes. To tackle this challenge, a crucial step is to aggregate target predictions to assign class weights by up-weighing target classes and down-weighing outlier classes. However, the incorrect predictions of class weights can mislead the network and lead to negative transfer. Previous works improve the class weight accuracy by utilizing temporal features and attention mechanisms, but these methods may fall short when trying to generate accurate class weight when domain shifts are significant, as in most real-world scenarios. To deal with these challenges, we propose the Multi-modality Cluster-calibrated partial Adversarial Network (MCAN). MCAN enhances video feature extraction with multi-modal features from multiple temporal scales to form more robust overall features. It utilizes a novel class weight calibration method to alleviate the negative transfer caused by incorrect class weights. The calibration method tries to identify and weigh correct and incorrect predictions using distributional information implied by unsupervised clustering. Extensive experiments are conducted on prevailing PVDA benchmarks, and the proposed MCAN achieves significant improvements when compared to state-of-the-art PVDA methods.