# Multi-level Contrast Network for Wearables-based Joint Activity Segmentation and Recognition

arXiv: https://arxiv.org/abs/2208.07547 · HTML: https://mlsanity.com/paper/2208.07547/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 33 references that are in the map.
- First posted on arXiv 16 Aug 2022.

## Builds on (most cited first, 17 of 17)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Hard Negative Mixing for Contrastive Learning](https://mlsanity.com/paper/2010.01028/) (2020)
- [MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation](https://mlsanity.com/paper/1903.01945/) (2019)
- [Time-Series Representation Learning via Temporal and Contextual Contrasting](https://mlsanity.com/paper/2106.14112/) (2021)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [Exploring Cross-Image Pixel Contrast for Semantic Segmentation](https://mlsanity.com/paper/2101.11939/) (2021)
- [Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities](https://mlsanity.com/paper/2001.07416/) (2020)
- [Ensembles of Deep LSTM Learners for Activity Recognition using Wearables](https://mlsanity.com/paper/1703.09370/) (2017)
- [Time Series Change Point Detection with Self-Supervised Contrastive Predictive Coding](https://mlsanity.com/paper/2011.14097/) (2020)
- [Attend And Discriminate: Beyond the State-of-the-Art for Human Activity Recognition using Wearable Sensors](https://mlsanity.com/paper/2007.07172/) (2020)
- [Multi-modal Fusion for Single-Stage Continuous Gesture Recognition](https://mlsanity.com/paper/2011.04945/) (2020)
- [MARS: Mixed Virtual and Real Wearable Sensors for Human Activity Recognition with Multi-Domain Deep Learning Model](https://mlsanity.com/paper/2009.09404/) (2020)
- [CreativeBioMan: Brain and Body Wearable Computing based Creative Gaming System](https://mlsanity.com/paper/1906.01801/) (2019)

## Built on by (most cited first, 2 of 2)

- [Timestamp-supervised Wearable-based Activity Segmentation and Recognition with Contrastive Learning and Order-Preserving Optimal Transport](https://mlsanity.com/paper/2310.09114/) (2023)
- [Suite-IN: Aggregating Motion Features from Apple Suite for Robust Inertial Navigation](https://mlsanity.com/paper/2411.07828/) (2024)

## Abstract (arXiv)

Human activity recognition (HAR) with wearables is promising research that can be widely adopted in many smart healthcare applications. In recent years, the deep learning-based HAR models have achieved impressive recognition performance. However, most HAR algorithms are susceptible to the multi-class windows problem that is essential yet rarely exploited. In this paper, we propose to relieve this challenging problem by introducing the segmentation technology into HAR, yielding joint activity segmentation and recognition. Especially, we introduce the Multi-Stage Temporal Convolutional Network (MS-TCN) architecture for sample-level activity prediction to joint segment and recognize the activity sequence. Furthermore, to enhance the robustness of HAR against the inter-class similarity and intra-class heterogeneity, a multi-level contrastive loss, containing the sample-level and segment-level contrast, has been proposed to learn a well-structured embedding space for better activity segmentation and recognition performance. Finally, with comprehensive experiments, we verify the effectiveness of the proposed method on two public HAR datasets, achieving significant improvements in the various evaluation metrics.