# Multimodal In-bed Pose and Shape Estimation under the Blankets

arXiv: https://arxiv.org/abs/2012.06735 · HTML: https://mlsanity.com/paper/2012.06735/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Dec 2020.

## Built on by (most cited first, 15 of 15)

- [BodyPressure -- Inferring Body Pose and Contact Pressure from a Depth Image](https://mlsanity.com/paper/2105.09936/) (2021)
- [Anatomy-guided domain adaptation for 3D in-bed human pose estimation](https://mlsanity.com/paper/2211.12193/) (2022)
- [In-Bed Human Pose Estimation from Unseen and Privacy-Preserving Image Domains](https://mlsanity.com/paper/2111.15124/) (2021)
- [Privacy-Preserving In-Bed Pose Monitoring: A Fusion and Reconstruction Study](https://mlsanity.com/paper/2202.10704/) (2022)
- [Under the Cover Infant Pose Estimation using Multimodal Data](https://mlsanity.com/paper/2210.00662/) (2022)
- [DiSRT-In-Bed: Diffusion-Based Sim-to-Real Transfer Framework for In-Bed Human Mesh Recovery](https://mlsanity.com/paper/2504.03006/) (2025)
- [BodyMAP -- Jointly Predicting Body Mesh and 3D Applied Pressure Map for People in Bed](https://mlsanity.com/paper/2404.03183/) (2024)
- [Self-supervised 3D Patient Modeling with Multi-modal Attentive Fusion](https://mlsanity.com/paper/2403.03217/) (2024)
- [PI-HMR: Towards Robust In-bed Temporal Human Shape Reconstruction with Contact Pressure Sensing](https://mlsanity.com/paper/2503.00068/) (2025)
- [PressMimic: Pressure-Guided Motion Capture and Control for Humanoid Robot Imitation](https://mlsanity.com/paper/2606.26741/) (2026)
- [In-Bed Pose Estimation: A Review](https://mlsanity.com/paper/2402.00700/) (2024)
- [Multi-modal 3D Pose and Shape Estimation with Computed Tomography](https://mlsanity.com/paper/2503.19405/) (2025)
- [Pressure2Motion: Hierarchical Human Motion Reconstruction from Ground Pressure with Text Guidance](https://mlsanity.com/paper/2511.05038/) (2025)
- [Improving the Plausibility of Pressure Distributions Synthesized from Depth Image through Generative Modeling](https://mlsanity.com/paper/2512.13757/) (2025)
- [Geometry-Conditioned Diffusion for Occlusion-Robust In-Bed Pose Estimation](https://mlsanity.com/paper/2604.23651/) (2026)

## Abstract (arXiv)

Humans spend vast hours in bed -- about one-third of the lifetime on average. Besides, a human at rest is vital in many healthcare applications. Typically, humans are covered by a blanket when resting, for which we propose a multimodal approach to uncover the subjects so their bodies at rest can be viewed without the occlusion of the blankets above. We propose a pyramid scheme to effectively fuse the different modalities in a way that best leverages the knowledge captured by the multimodal sensors. Specifically, the two most informative modalities (i.e., depth and infrared images) are first fused to generate good initial pose and shape estimation. Then pressure map and RGB images are further fused one by one to refine the result by providing occlusion-invariant information for the covered part, and accurate shape information for the uncovered part, respectively. However, even with multimodal data, the task of detecting human bodies at rest is still very challenging due to the extreme occlusion of bodies. To further reduce the negative effects of the occlusion from blankets, we employ an attention-based reconstruction module to generate uncovered modalities, which are further fused to update current estimation via a cyclic fashion. Extensive experiments validate the superiority of the proposed model over others.