# Virtual Fusion with Contrastive Learning for Single Sensor-based Activity Recognition

arXiv: https://arxiv.org/abs/2312.02185 · HTML: https://mlsanity.com/paper/2312.02185/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 43 references that are in the map.
- First posted on arXiv 1 Dec 2023.

## Builds on (most cited first, 10 of 10)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2010.04592/) (2020)
- [Data Augmentation of Wearable Sensor Data for Parkinson's Disease Monitoring using Convolutional Neural Networks](https://mlsanity.com/paper/1706.00527/) (2017)
- [Boosting Contrastive Self-Supervised Learning with False Negative Cancellation](https://mlsanity.com/paper/2011.11765/) (2020)
- [DanHAR: Dual Attention Network For Multimodal Human Activity Recognition Using Wearable Sensors](https://mlsanity.com/paper/2006.14435/) (2020)
- [Contrastive Learning with Cross-Modal Knowledge Mining for Multimodal Human Activity Recognition](https://mlsanity.com/paper/2205.10071/) (2022)
- [Multi Visual Modality Fall Detection Dataset](https://mlsanity.com/paper/2206.12740/) (2022)
- [Negative Selection by Clustering for Contrastive Learning in Human Activity Recognition](https://mlsanity.com/paper/2203.12230/) (2022)

## Built on by (most cited first, 9 of 9)

- [Beyond Confusion: A Fine-grained Dialectical Examination of Human Activity Recognition Benchmark Datasets](https://mlsanity.com/paper/2412.09037/) (2024)
- [Scaling laws in wearable human activity recognition](https://mlsanity.com/paper/2502.03364/) (2025)
- [SoK: Behind the Accuracy of Complex Human Activity Recognition Using Deep Learning](https://mlsanity.com/paper/2405.00712/) (2024)
- [ALS-HAR: Harnessing Wearable Ambient Light Sensors to Enhance IMU-based Human Activity Recogntion](https://mlsanity.com/paper/2408.09527/) (2024)
- [REMONI: An Autonomous System Integrating Wearables and Multimodal Large Language Models for Enhanced Remote Health Monitoring](https://mlsanity.com/paper/2510.21445/) (2025)
- [Align and Adapt: Multimodal Multiview Human Activity Recognition under Arbitrary View Combinations](https://mlsanity.com/paper/2602.08755/) (2026)
- [Structure over Depth: A Single-Block Spatio-Temporal Transformer for Multi-Entity Reasoning](https://mlsanity.com/paper/2607.23077/) (2026)
- [Omni-modal decomposition autoencoders learn full-stack wearable disentangled representations](https://mlsanity.com/paper/2608.07385/) (2026)
- [Dynamic Influence-Weighted Distillation for Single-IMU Activity Recognition](https://mlsanity.com/paper/2608.24904/) (2026)

## Abstract (arXiv)

Various types of sensors can be used for Human Activity Recognition (HAR), and each of them has different strengths and weaknesses. Sometimes a single sensor cannot fully observe the user's motions from its perspective, which causes wrong predictions. While sensor fusion provides more information for HAR, it comes with many inherent drawbacks like user privacy and acceptance, costly set-up, operation, and maintenance. To deal with this problem, we propose Virtual Fusion - a new method that takes advantage of unlabeled data from multiple time-synchronized sensors during training, but only needs one sensor for inference. Contrastive learning is adopted to exploit the correlation among sensors. Virtual Fusion gives significantly better accuracy than training with the same single sensor, and in some cases, it even surpasses actual fusion using multiple sensors at test time. We also extend this method to a more general version called Actual Fusion within Virtual Fusion (AFVF), which uses a subset of training sensors during inference. Our method achieves state-of-the-art accuracy and F1-score on UCI-HAR and PAMAP2 benchmark datasets. Implementation is available upon request.