# Federated Self-Supervised Learning of Multi-Sensor Representations for Embedded Intelligence

arXiv: https://arxiv.org/abs/2007.13018 · HTML: https://mlsanity.com/paper/2007.13018/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jul 2020.

## Built on by (most cited first, 30 of 40)

- [Assessing the State of Self-Supervised Human Activity Recognition using Wearables](https://mlsanity.com/paper/2202.12938/) (2022)
- [FedMultimodal: A Benchmark For Multimodal Federated Learning](https://mlsanity.com/paper/2306.09486/) (2023)
- [Multimodal Federated Learning on IoT Data](https://mlsanity.com/paper/2109.04833/) (2021)
- [Self-supervised Contrastive Learning for EEG-based Sleep Staging](https://mlsanity.com/paper/2109.07839/) (2021)
- [SSFL: Tackling Label Deficiency in Federated Learning via Personalized Self-Supervision](https://mlsanity.com/paper/2110.02470/) (2021)
- [Beyond Just Vision: A Review on Self-Supervised Representation Learning on Multimodal and Temporal Data](https://mlsanity.com/paper/2206.02353/) (2022)
- [ColloSSL: Collaborative Self-Supervised Learning for Human Activity Recognition](https://mlsanity.com/paper/2202.00758/) (2022)
- [Federated Learning for Computer Vision](https://mlsanity.com/paper/2308.13558/) (2023)
- [COCOA: Cross Modality Contrastive Learning for Sensor Data](https://mlsanity.com/paper/2208.00467/) (2022)
- [Exploring Self-Supervised Representation Ensembles for COVID-19 Cough Classification](https://mlsanity.com/paper/2105.07566/) (2021)
- [Autonomy and Intelligence in the Computing Continuum: Challenges, Enablers, and Future Directions for Orchestration](https://mlsanity.com/paper/2205.01423/) (2022)
- [A Survey on Federated Learning and its Applications for Accelerating Industrial Internet of Things](https://mlsanity.com/paper/2104.10501/) (2021)
- [Personalized Semi-Supervised Federated Learning for Human Activity Recognition](https://mlsanity.com/paper/2104.08094/) (2021)
- [Sensor Data Augmentation by Resampling for Contrastive Learning in Human Activity Recognition](https://mlsanity.com/paper/2109.02054/) (2021)
- [SemiPFL: Personalized Semi-Supervised Federated Learning Framework for Edge Intelligence](https://mlsanity.com/paper/2203.08176/) (2022)
- [Self-Supervised WiFi-Based Activity Recognition](https://mlsanity.com/paper/2104.09072/) (2021)
- [Self-supervised Group Meiosis Contrastive Learning for EEG-Based Emotion Recognition](https://mlsanity.com/paper/2208.00877/) (2022)
- [Towards Generalizable Human Activity Recognition: A Survey](https://mlsanity.com/paper/2508.12213/) (2025)
- [Federated Self-Supervised Learning for Acoustic Event Classification](https://mlsanity.com/paper/2203.11997/) (2022)
- [A Comprehensive Survey on Federated Learning: Concept and Applications](https://mlsanity.com/paper/2201.09384/) (2022)
- [Self-Supervised Human Activity Recognition with Localized Time-Frequency Contrastive Representation Learning](https://mlsanity.com/paper/2209.00990/) (2022)
- [Distributed Traffic Synthesis and Classification in Edge Networks: A Federated Self-supervised Learning Approach](https://mlsanity.com/paper/2302.00207/) (2023)
- [DeepFIB: Self-Imputation for Time Series Anomaly Detection](https://mlsanity.com/paper/2112.06247/) (2021)
- [Self-Supervised Learning for WiFi CSI-Based Human Activity Recognition: A Systematic Study](https://mlsanity.com/paper/2308.02412/) (2023)
- [Disentangling the Effects of Data Augmentation and Format Transform in Self-Supervised Learning of Image Representations](https://mlsanity.com/paper/2312.02205/) (2023)
- [Exploring Federated Self-Supervised Learning for General Purpose Audio Understanding](https://mlsanity.com/paper/2402.02889/) (2024)
- [Wearable-based Classification of Running Styles with Deep Learning](https://mlsanity.com/paper/2109.00594/) (2021)
- [Federated Self-Supervised Learning in Heterogeneous Settings: Limits of a Baseline Approach on HAR](https://mlsanity.com/paper/2207.08187/) (2022)
- [IoT Federated Blockchain Learning at the Edge](https://mlsanity.com/paper/2304.03006/) (2023)
- [Federated Learning on Edge Sensing Devices: A Review](https://mlsanity.com/paper/2311.01201/) (2023)

## Abstract (arXiv)

Smartphones, wearables, and Internet of Things (IoT) devices produce a wealth of data that cannot be accumulated in a centralized repository for learning supervised models due to privacy, bandwidth limitations, and the prohibitive cost of annotations. Federated learning provides a compelling framework for learning models from decentralized data, but conventionally, it assumes the availability of labeled samples, whereas on-device data are generally either unlabeled or cannot be annotated readily through user interaction. To address these issues, we propose a self-supervised approach termed textitscalogram-signal correspondence learning based on wavelet transform to learn useful representations from unlabeled sensor inputs, such as electroencephalography, blood volume pulse, accelerometer, and WiFi channel state information. Our auxiliary task requires a deep temporal neural network to determine if a given pair of a signal and its complementary viewpoint (i.e., a scalogram generated with a wavelet transform) align with each other or not through optimizing a contrastive objective. We extensively assess the quality of learned features with our multi-view strategy on diverse public datasets, achieving strong performance in all domains. We demonstrate the effectiveness of representations learned from an unlabeled input collection on downstream tasks with training a linear classifier over pretrained network, usefulness in low-data regime, transfer learning, and cross-validation. Our methodology achieves competitive performance with fully-supervised networks, and it outperforms pre-training with autoencoders in both central and federated contexts. Notably, it improves the generalization in a semi-supervised setting as it reduces the volume of labeled data required through leveraging self-supervised learning.