# TS-MoCo: Time-Series Momentum Contrast for Self-Supervised Physiological Representation Learning

arXiv: https://arxiv.org/abs/2306.06522 · HTML: https://mlsanity.com/paper/2306.06522/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 21 references that are in the map.
- First posted on arXiv 10 Jun 2023.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Time-Series Representation Learning via Temporal and Contextual Contrasting](https://mlsanity.com/paper/2106.14112/) (2021)
- [ColloSSL: Collaborative Self-Supervised Learning for Human Activity Recognition](https://mlsanity.com/paper/2202.00758/) (2022)
- [GANSER: A Self-supervised Data Augmentation Framework for EEG-based Emotion Recognition](https://mlsanity.com/paper/2109.03124/) (2021)
- [EEG2Vec: Learning Affective EEG Representations via Variational Autoencoders](https://mlsanity.com/paper/2207.08002/) (2022)
- [Domain-Invariant Representation Learning from EEG with Private Encoders](https://mlsanity.com/paper/2201.11613/) (2022)
- [Universal Physiological Representation Learning with Soft-Disentangled Rateless Autoencoders](https://mlsanity.com/paper/2009.13453/) (2020)
- [Exploiting Multiple EEG Data Domains with Adversarial Learning](https://mlsanity.com/paper/2204.07777/) (2022)
- [Self-Supervised Learning as a Means To Reduce the Need for Labeled Data in Medical Image Analysis](https://mlsanity.com/paper/2206.00344/) (2022)
- [Stochastic Mutual Information Gradient Estimation for Dimensionality Reduction Networks](https://mlsanity.com/paper/2105.00191/) (2021)

## Built on by (most cited first, 1 of 1)

- [Self-supervised Learning for Electroencephalogram: A Systematic Survey](https://mlsanity.com/paper/2401.05446/) (2024)

## Abstract (arXiv)

Limited availability of labeled physiological data often prohibits the use of powerful supervised deep learning models in the biomedical machine intelligence domain. We approach this problem and propose a novel encoding framework that relies on self-supervised learning with momentum contrast to learn representations from multivariate time-series of various physiological domains without needing labels. Our model uses a transformer architecture that can be easily adapted to classification problems by optimizing a linear output classification layer. We experimentally evaluate our framework using two publicly available physiological datasets from different domains, i.e., human activity recognition from embedded inertial sensory and emotion recognition from electroencephalography. We show that our self-supervised learning approach can indeed learn discriminative features which can be exploited in downstream classification tasks. Our work enables the development of domain-agnostic intelligent systems that can effectively analyze multivariate time-series data from physiological domains.