# MuSACo: Multimodal Subject-Specific Selection and Adaptation for Expression Recognition with Co-Training

arXiv: https://arxiv.org/abs/2508.12522 · HTML: https://mlsanity.com/paper/2508.12522/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 54 references that are in the map.
- First posted on arXiv 17 Aug 2025.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition](https://mlsanity.com/paper/1912.10211/) (2019)
- [Unsupervised Domain Adaptation with Residual Transfer Networks](https://mlsanity.com/paper/1602.04433/) (2016)
- [Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift](https://mlsanity.com/paper/1803.00830/) (2018)
- [Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation](https://mlsanity.com/paper/2006.16806/) (2020)
- [End-to-End Multimodal Emotion Recognition using Deep Neural Networks](https://mlsanity.com/paper/1704.08619/) (2017)
- [3D Semi-Supervised Learning with Uncertainty-Aware Multi-View Co-Training](https://mlsanity.com/paper/1811.12506/) (2018)
- [Your Classifier can Secretly Suffice Multi-Source Domain Adaptation](https://mlsanity.com/paper/2103.11169/) (2021)
- [Deep Co-Training with Task Decomposition for Semi-Supervised Domain Adaptation](https://mlsanity.com/paper/2007.12684/) (2020)
- [A Joint Cross-Attention Model for Audio-Visual Fusion in Dimensional Emotion Recognition](https://mlsanity.com/paper/2203.14779/) (2022)
- [Multi-Source Unsupervised Domain Adaptation via Pseudo Target Domain](https://mlsanity.com/paper/2202.10725/) (2022)
- [MADAN: Multi-source Adversarial Domain Aggregation Network for Domain Adaptation](https://mlsanity.com/paper/2003.00820/) (2020)
- [A Differential Entropy Estimator for Training Neural Networks](https://mlsanity.com/paper/2202.06618/) (2022)
- [Distilling Privileged Multimodal Information for Expression Recognition using Optimal Transport](https://mlsanity.com/paper/2401.15489/) (2024)
- [Joint Multimodal Transformer for Emotion Recognition in the Wild](https://mlsanity.com/paper/2403.10488/) (2024)
- [Multi-Source domain adaptation via supervised contrastive learning and confident consistency regularization](https://mlsanity.com/paper/2106.16093/) (2021)
- [Deep Weakly-Supervised Domain Adaptation for Pain Localization in Videos](https://mlsanity.com/paper/1910.08173/) (2019)
- [Deep Domain Adaptation for Ordinal Regression of Pain Intensity Estimation Using Weakly-Labelled Videos](https://mlsanity.com/paper/2008.06392/) (2020)
- [Disentangling Identity and Pose for Facial Expression Recognition](https://mlsanity.com/paper/2208.08106/) (2022)
- [Subject-Based Domain Adaptation for Facial Expression Recognition](https://mlsanity.com/paper/2312.05632/) (2023)
- [Source-Free Domain Adaptation of Weakly-Supervised Object Localization Models for Histology](https://mlsanity.com/paper/2404.19113/) (2024)
- [Disentangled Source-Free Personalization for Facial Expression Recognition with Neutral Target Data](https://mlsanity.com/paper/2503.20771/) (2025)
- [Co-Training for Unsupervised Domain Adaptation of Semantic Segmentation Models](https://mlsanity.com/paper/2205.15781/) (2022)
- [Progressive Multi-Source Domain Adaptation for Personalized Facial Expression Recognition](https://mlsanity.com/paper/2504.04252/) (2025)
- [Robust Target Training for Multi-Source Domain Adaptation](https://mlsanity.com/paper/2210.01676/) (2022)

## Built on by (most cited first, 4 of 4)

- [Test-Time Adaptation via Cache Personalization for Facial Expression Recognition in Videos](https://mlsanity.com/paper/2603.21309/) (2026)
- [CLIP-AUTT: Test-Time Personalization with Action Unit Prompting for Fine-Grained Video Emotion Recognition](https://mlsanity.com/paper/2603.27999/) (2026)
- [Selective Prediction Forgetting for Adaptation of Weakly Supervised Localization of Histology Images](https://mlsanity.com/paper/2603.12468/) (2026)
- [Test-Time Adaptation with Online Personalized Energy-Based Cache for Fine-Grained Video Expression Recognition](https://mlsanity.com/paper/2608.06467/) (2026)

## Abstract (arXiv)

Personalized expression recognition (ER) involves adapting a machine learning model to subject-specific data for improved recognition of expressions with considerable interpersonal variability. Subject-specific ER can benefit significantly from multi-source domain adaptation (MSDA) methods, where each domain corresponds to a specific subject to improve model accuracy and robustness. Despite promising results, state-of-the-art MSDA approaches often overlook multimodal information or blend sources into a single domain, limiting subject diversity and failing to explicitly capture unique subject-specific characteristics. To address these limitations, we introduce MuSACo, a multimodal subject-specific selection and adaptation method for ER based on co-training. It leverages complementary information across multiple modalities and multiple source domains for subject-specific adaptation. This makes MuSACo particularly relevant for affective computing applications in digital health, such as patient-specific assessment for stress or pain, where subject-level nuances are crucial. MuSACo selects source subjects relevant to the target and generates pseudo-labels using the dominant modality for class-aware learning, in conjunction with a class-agnostic loss to learn from less confident target samples. Finally, source features from each modality are aligned, while only confident target features are combined. Experimental results on challenging multimodal ER datasets: BioVid, StressID, and BAH show that MuSACo outperforms UDA (blending) and state-of-the-art MSDA methods.