# SelfReplay: Adapting Self-Supervised Sensory Models via Adaptive Meta-Task Replay

arXiv: https://arxiv.org/abs/2404.15305 · HTML: https://mlsanity.com/paper/2404.15305/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 70 references that are in the map.
- First posted on arXiv 29 Mar 2024.

## Builds on (most cited first, 30 of 36)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/1902.06162/) (2019)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Domain Generalization with MixStyle](https://mlsanity.com/paper/2104.02008/) (2021)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [A Survey of Unsupervised Deep Domain Adaptation](https://mlsanity.com/paper/1812.02849/) (2018)
- [Learning to Learn Single Domain Generalization](https://mlsanity.com/paper/2003.13216/) (2020)
- [Episodic Training for Domain Generalization](https://mlsanity.com/paper/1902.00113/) (2019)
- [SelfReg: Self-supervised Contrastive Regularization for Domain Generalization](https://mlsanity.com/paper/2104.09841/) (2021)
- [Heterogeneous Domain Generalization via Domain Mixup](https://mlsanity.com/paper/2009.05448/) (2020)
- [Multi-task Self-Supervised Learning for Human Activity Detection](https://mlsanity.com/paper/1907.11879/) (2019)
- [AdCo: Adversarial Contrast for Efficient Learning of Unsupervised Representations from Self-Trained Negative Adversaries](https://mlsanity.com/paper/2011.08435/) (2020)
- [Unsupervised Meta-Learning For Few-Shot Image Classification](https://mlsanity.com/paper/1811.11819/) (2018)
- [Exploring Contrastive Learning in Human Activity Recognition for Healthcare](https://mlsanity.com/paper/2011.11542/) (2020)
- [Self-supervised Learning for Human Activity Recognition Using 700,000 Person-days of Wearable Data](https://mlsanity.com/paper/2206.02909/) (2022)
- [Correlation-aware Adversarial Domain Adaptation and Generalization](https://mlsanity.com/paper/1911.12983/) (2019)
- [Assessing the State of Self-Supervised Human Activity Recognition using Wearables](https://mlsanity.com/paper/2202.12938/) (2022)
- [Contrastive Predictive Coding for Human Activity Recognition](https://mlsanity.com/paper/2012.05333/) (2020)
- [Stratified Transfer Learning for Cross-domain Activity Recognition](https://mlsanity.com/paper/1801.00820/) (2017)
- [Semantic-Discriminative Mixup for Generalizable Sensor-based Cross-domain Activity Recognition](https://mlsanity.com/paper/2206.06629/) (2022)
- [Unsupervised Meta-Learning through Latent-Space Interpolation in Generative Models](https://mlsanity.com/paper/2006.10236/) (2020)
- [ColloSSL: Collaborative Self-Supervised Learning for Human Activity Recognition](https://mlsanity.com/paper/2202.00758/) (2022)
- [Cross-domain Activity Recognition via Substructural Optimal Transport](https://mlsanity.com/paper/2102.03353/) (2021)
- [COCOA: Cross Modality Contrastive Learning for Sensor Data](https://mlsanity.com/paper/2208.00467/) (2022)

## Built on by (most cited first, 1 of 1)

- [Multimodal Federated Learning under Dual-Axis Modality Missingness](https://mlsanity.com/paper/2608.09240/) (2026)

## Abstract (arXiv)

Self-supervised learning has emerged as a method for utilizing massive unlabeled data for pre-training models, providing an effective feature extractor for various mobile sensing applications. However, when deployed to end-users, these models encounter significant domain shifts attributed to user diversity. We investigate the performance degradation that occurs when self-supervised models are fine-tuned in heterogeneous domains. To address the issue, we propose SelfReplay, a few-shot domain adaptation framework for personalizing self-supervised models. SelfReplay proposes self-supervised meta-learning for initial model pre-training, followed by a user-side model adaptation by replaying the self-supervision with user-specific data. This allows models to adjust their pre-trained representations to the user with only a few samples. Evaluation with four benchmarks demonstrates that SelfReplay outperforms existing baselines by an average F1-score of 8.8%p. Our on-device computational overhead analysis on a commodity off-the-shelf (COTS) smartphone shows that SelfReplay completes adaptation within an unobtrusive latency (in three minutes) with only a 9.54% memory consumption, demonstrating the computational efficiency of the proposed method.