# Simultaneously-Collected Multimodal Lying Pose Dataset: Towards In-Bed Human Pose Monitoring under Adverse Vision Conditions

arXiv: https://arxiv.org/abs/2008.08735 · HTML: https://mlsanity.com/paper/2008.08735/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Aug 2020.

## Built on by (most cited first, 9 of 9)

- [Object Detection Using Sim2Real Domain Randomization for Robotic Applications](https://mlsanity.com/paper/2208.04171/) (2022)
- [BodyPressure -- Inferring Body Pose and Contact Pressure from a Depth Image](https://mlsanity.com/paper/2105.09936/) (2021)
- [In-Bed Human Pose Estimation from Unseen and Privacy-Preserving Image Domains](https://mlsanity.com/paper/2111.15124/) (2021)
- [Towards Accurate Cross-Domain In-Bed Human Pose Estimation](https://mlsanity.com/paper/2110.03578/) (2021)
- [Privacy-Preserving In-Bed Pose Monitoring: A Fusion and Reconstruction Study](https://mlsanity.com/paper/2202.10704/) (2022)
- [Pressure Eye: In-bed Contact Pressure Estimation via Contact-less Imaging](https://mlsanity.com/paper/2201.11828/) (2022)
- [Human Pose Estimation from Ambiguous Pressure Recordings with Spatio-temporal Masked Transformers](https://mlsanity.com/paper/2303.05691/) (2023)
- [MassNet: A Deep Learning Approach for Body Weight Extraction from A Single Pressure Image](https://mlsanity.com/paper/2303.10136/) (2023)
- [A Self-supervised Pressure Map human keypoint Detection Approch: Optimizing Generalization and Computational Efficiency Across Datasets](https://mlsanity.com/paper/2402.14241/) (2024)

## Abstract (arXiv)

Computer vision (CV) has achieved great success in interpreting semantic meanings from images, yet CV algorithms can be brittle for tasks with adverse vision conditions and the ones suffering from data/label pair limitation. One of this tasks is in-bed human pose estimation, which has significant values in many healthcare applications. In-bed pose monitoring in natural settings could involve complete darkness or full occlusion. Furthermore, the lack of publicly available in-bed pose datasets hinders the use of many successful pose estimation algorithms for this task. In this paper, we introduce our Simultaneously-collected multimodal Lying Pose (SLP) dataset, which includes in-bed pose images from 109 participants captured using multiple imaging modalities including RGB, long wave infrared, depth, and pressure map. We also present a physical hyper parameter tuning strategy for ground truth pose label generation under extreme conditions such as lights off and being fully covered by a sheet/blanket. SLP design is compatible with the mainstream human pose datasets, therefore, the state-of-the-art 2D pose estimation models can be trained effectively with SLP data with promising performance as high as 95% at PCKh@0.5 on a single modality. The pose estimation performance can be further improved by including additional modalities through collaboration.