# Privacy-Preserving In-Bed Pose Monitoring: A Fusion and Reconstruction Study

arXiv: https://arxiv.org/abs/2202.10704 · HTML: https://mlsanity.com/paper/2202.10704/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 43 references that are in the map.
- First posted on arXiv 22 Feb 2022.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [Efficient Object Localization Using Convolutional Networks](https://mlsanity.com/paper/1411.4280/) (2014)
- [Monocular Human Pose Estimation: A Survey of Deep Learning-based Methods](https://mlsanity.com/paper/2006.01423/) (2020)
- [Flowing ConvNets for Human Pose Estimation in Videos](https://mlsanity.com/paper/1506.02897/) (2015)
- [Seeing Under the Cover: A Physics Guided Learning Approach for In-Bed Pose Estimation](https://mlsanity.com/paper/1907.02161/) (2019)
- [Multimodal In-bed Pose and Shape Estimation under the Blankets](https://mlsanity.com/paper/2012.06735/) (2020)
- [In-Bed Pose Estimation: Deep Learning with Shallow Dataset](https://mlsanity.com/paper/1711.01005/) (2017)
- [Simultaneously-Collected Multimodal Lying Pose Dataset: Towards In-Bed Human Pose Monitoring under Adverse Vision Conditions](https://mlsanity.com/paper/2008.08735/) (2020)
- [In-Bed Human Pose Estimation from Unseen and Privacy-Preserving Image Domains](https://mlsanity.com/paper/2111.15124/) (2021)
- [Im2Mesh GAN: Accurate 3D Hand Mesh Recovery from a Single RGB Image](https://mlsanity.com/paper/2101.11239/) (2021)
- [Video-Based Inpatient Fall Risk Assessment: A Case Study](https://mlsanity.com/paper/2106.07565/) (2021)

## Built on by (most cited first, 6 of 6)

- [Under the Cover Infant Pose Estimation using Multimodal Data](https://mlsanity.com/paper/2210.00662/) (2022)
- [MiPa: Mixed Patch Infrared-Visible Modality Agnostic Object Detection](https://mlsanity.com/paper/2404.18849/) (2024)
- [In-Bed Pose Estimation: A Review](https://mlsanity.com/paper/2402.00700/) (2024)
- [BlanketGen2-Fit3D: Synthetic Blanket Augmentation Towards Improving Real-World In-Bed Blanket Occluded Human Pose Estimation](https://mlsanity.com/paper/2501.12318/) (2025)
- [Improving the Plausibility of Pressure Distributions Synthesized from Depth Image through Generative Modeling](https://mlsanity.com/paper/2512.13757/) (2025)
- [Geometry-Conditioned Diffusion for Occlusion-Robust In-Bed Pose Estimation](https://mlsanity.com/paper/2604.23651/) (2026)

## Abstract (arXiv)

Recently, in-bed human pose estimation has attracted the interest of researchers due to its relevance to a wide range of healthcare applications. Compared to the general problem of human pose estimation, in-bed pose estimation has several inherent challenges, the most prominent being frequent and severe occlusions caused by bedding. In this paper we explore the effective use of images from multiple non-visual and privacy-preserving modalities such as depth, long-wave infrared (LWIR) and pressure maps for the task of in-bed pose estimation in two settings. First, we explore the effective fusion of information from different imaging modalities for better pose estimation. Secondly, we propose a framework that can estimate in-bed pose estimation when visible images are unavailable, and demonstrate the applicability of fusion methods to scenarios where only LWIR images are available. We analyze and demonstrate the effect of fusing features from multiple modalities. For this purpose, we consider four different techniques: 1) Addition, 2) Concatenation, 3) Fusion via learned modal weights, and 4) End-to-end fully trainable approach; with a state-of-the-art pose estimation model. We also evaluate the effect of reconstructing a data-rich modality (i.e., visible modality) from a privacy-preserving modality with data scarcity (i.e., long-wavelength infrared) for in-bed human pose estimation. For reconstruction, we use a conditional generative adversarial network. We conduct ablative studies across different design decisions of our framework. This includes selecting features with different levels of granularity, using different fusion techniques, and varying model parameters. Through extensive evaluations, we demonstrate that our method produces on par or better results compared to the state-of-the-art.