# M3BAT: Unsupervised Domain Adaptation for Multimodal Mobile Sensing with Multi-Branch Adversarial Training

arXiv: https://arxiv.org/abs/2404.17391 · HTML: https://mlsanity.com/paper/2404.17391/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 92 references that are in the map.
- First posted on arXiv 26 Apr 2024.

## Builds on (most cited first, 30 of 34)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Contrastive Adaptation Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1901.00976/) (2019)
- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark](https://mlsanity.com/paper/2109.14545/) (2021)
- [Self-Supervised Learning for Domain Adaptation on Point-Clouds](https://mlsanity.com/paper/2003.12641/) (2020)
- [Unsupervised domain adaptation for medical imaging segmentation with self-ensembling](https://mlsanity.com/paper/1811.06042/) (2018)
- [Beyond Sharing Weights for Deep Domain Adaptation](https://mlsanity.com/paper/1603.06432/) (2016)
- [Domain Adaptation for Time Series Under Feature and Label Shifts](https://mlsanity.com/paper/2302.03133/) (2023)
- [Self-supervised Domain Adaptation for Computer Vision Tasks](https://mlsanity.com/paper/1907.10915/) (2019)
- [Curriculum Manager for Source Selection in Multi-Source Domain Adaptation](https://mlsanity.com/paper/2007.01261/) (2020)
- [Contrastive Learning for Unsupervised Domain Adaptation of Time Series](https://mlsanity.com/paper/2206.06243/) (2022)
- [Learning from the Best: Contrastive Representations Learning Across Sensor Locations for Wearable Activity Recognition](https://mlsanity.com/paper/2210.01459/) (2022)
- [Generalization and Personalization of Mobile Sensing-Based Mood Inference Models: An Analysis of College Students in Eight Countries](https://mlsanity.com/paper/2211.03009/) (2022)
- [Smartphone Sensing for the Well-being of Young Adults: A Review](https://mlsanity.com/paper/2012.09559/) (2020)
- [DAPPER: Label-Free Performance Estimation after Personalization for Heterogeneous Mobile Sensing](https://mlsanity.com/paper/2111.11053/) (2021)
- [Complex Daily Activities, Country-Level Diversity, and Smartphone Sensing: A Study in Denmark, Italy, Mongolia, Paraguay, and UK](https://mlsanity.com/paper/2302.08591/) (2023)
- [The Theory, Practice, and Ethical Challenges of Designing a Diversity-Aware Platform for Social Relations](https://mlsanity.com/paper/2202.12700/) (2022)
- [Understanding the Social Context of Eating with Multimodal Smartphone Sensing: The Role of Country Diversity](https://mlsanity.com/paper/2306.00709/) (2023)
- [Learning About Social Context from Smartphone Data: Generalization Across Countries and Daily Life Moments](https://mlsanity.com/paper/2306.00919/) (2023)
- [UDAMA: Unsupervised Domain Adaptation through Multi-discriminator Adversarial Training with Noisy Labels Improves Cardio-fitness Prediction](https://mlsanity.com/paper/2307.16651/) (2023)
- [Alone or With Others? Understanding Eating Episodes of College Students with Mobile Sensing](https://mlsanity.com/paper/2011.11694/) (2020)
- [Sensing Eating Events in Context: A Smartphone-Only Approach](https://mlsanity.com/paper/2205.14191/) (2022)
- [Inferring Mood-While-Eating with Smartphone Sensing and Community-Based Model Personalization](https://mlsanity.com/paper/2306.00723/) (2023)

## Built on by (most cited first, 8 of 8)

- [On-device Learning of EEGNet-based Network For Wearable Motor Imagery Brain-Computer Interface](https://mlsanity.com/paper/2409.00083/) (2024)
- [PromptTSS: A Prompting-Based Approach for Interactive Multi-Granularity Time Series Segmentation](https://mlsanity.com/paper/2506.11170/) (2025)
- [Human Heterogeneity Invariant Stress Sensing](https://mlsanity.com/paper/2506.02256/) (2025)
- [Chorus: Harmonizing Context and Sensing Signals for Data-Free Model Customization in IoT](https://mlsanity.com/paper/2512.15206/) (2025)
- [A Comparative Study of Traditional Machine Learning, Deep Learning, and Large Language Models for Mental Health Forecasting using Smartphone Sensing Data](https://mlsanity.com/paper/2601.03603/) (2026)
- [Perseus: Interactive Time Series Segmentation with Sparse Supervision via Stateful Memory](https://mlsanity.com/paper/2510.09930/) (2025)
- [Evaluating Federated Learning for Cross-Country Mood Inference from Smartphone Sensing Data](https://mlsanity.com/paper/2602.15478/) (2026)
- [Semi-Supervised Noise Adaptation: Transferring Knowledge from Noise Domain](https://mlsanity.com/paper/2606.00558/) (2026)

## Abstract (arXiv)

Over the years, multimodal mobile sensing has been used extensively for inferences regarding health and well being, behavior, and context. However, a significant challenge hindering the widespread deployment of such models in real world scenarios is the issue of distribution shift. This is the phenomenon where the distribution of data in the training set differs from the distribution of data in the real world, the deployment environment. While extensively explored in computer vision and natural language processing, and while prior research in mobile sensing briefly addresses this concern, current work primarily focuses on models dealing with a single modality of data, such as audio or accelerometer readings, and consequently, there is little research on unsupervised domain adaptation when dealing with multimodal sensor data. To address this gap, we did extensive experiments with domain adversarial neural networks (DANN) showing that they can effectively handle distribution shifts in multimodal sensor data. Moreover, we proposed a novel improvement over DANN, called M3BAT, unsupervised domain adaptation for multimodal mobile sensing with multi-branch adversarial training, to account for the multimodality of sensor data during domain adaptation with multiple branches. Through extensive experiments conducted on two multimodal mobile sensing datasets, three inference tasks, and 14 source-target domain pairs, including both regression and classification, we demonstrate that our approach performs effectively on unseen domains. Compared to directly deploying a model trained in the source domain to the target domain, the model shows performance increases up to 12% AUC (area under the receiver operating characteristics curves) on classification tasks, and up to 0.13 MAE (mean absolute error) on regression tasks.