mlsanityData to 7 Oct 2026

Multi-modal 3D Pose and Shape Estimation with Computed Tomography

arXiv 2503.19405 ↗First posted 25 Mar 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

In perioperative care, precise in-bed 3D patient pose and shape estimation (PSE) can be vital in optimizing patient positioning in preoperative planning, enabling accurate overlay of medical images for augmented reality-based surgical navigation, and mitigating risks of prolonged immobility during recovery. Conventional PSE methods relying on modalities such as RGB-D, infrared, or pressure maps often struggle with occlusions caused by bedding and complex patient positioning, leading to inaccurate estimation that can affect clinical outcomes. To address these challenges, we present the first multi-modal in-bed patient 3D PSE network that fuses detailed geometric features extracted from routinely acquired computed tomography (CT) scans with depth maps (mPSE-CT). mPSE-CT incorporates a shape estimation module that utilizes probabilistic correspondence alignment, a pose estimation module with a refined neural network, and a final parameters mixing module. This multi-modal network robustly reconstructs occluded body regions and enhances the accuracy of the estimated 3D human mesh model. We validated mPSE-CT using proprietary whole-body rigid phantom and volunteer datasets in clinical scenarios. mPSE-CT outperformed the best-performing prior method by 23% and 49.16% in pose and shape estimation respectively, demonstrating its potential for improving clinical outcomes in challenging perioperative environments.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 44 references that are in the map. First posted on arXiv 25 Mar 2025.

Builds on

19 of 44 references.

References · 19

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  3. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  4. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  5. 2018Simple Baselines for Human Pose Estimation and Trackingcited by 502
  6. 2018Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimationcited by 185
  7. 2022Accurate 3D Body Shape Regression using Metric and Semantic Attributescited by 49
  8. 2020AdaFuse: Adaptive Multiview Fusion for Accurate Human Pose Estimation in the Wildcited by 40
  9. 2020Bodies at Rest: 3D Human Pose and Shape Estimation from a Pressure Image using Synthetic Datacited by 34
  10. 2021BodyPressure -- Inferring Body Pose and Contact Pressure from a Depth Imagecited by 20
  11. 2019Seeing Under the Cover: A Physics Guided Learning Approach for In-Bed Pose Estimationcited by 17
  12. 2020Multimodal In-bed Pose and Shape Estimation under the Blanketscited by 15
7 more
  1. 20183D Human Pose Estimation on a Configurable Bed from a Pressure Imagecited by 13
  2. 2019RFBNet: Deep Multimodal Networks with Residual Fusion Blocks for RGB-D Semantic Segmentationcited by 12
  3. 2022Anatomy-guided domain adaptation for 3D in-bed human pose estimationcited by 9
  4. 2021Self-supervised 3D Human Mesh Recovery from Noisy Point Cloudscited by 8
  5. 2019In-bed Pressure-based Pose Estimation using Image Space Representation Learningcited by 7
  6. 2024BodyMAP -- Jointly Predicting Body Mesh and 3D Applied Pressure Map for People in Bedcited by 5
  7. 2024Self-supervised 3D Patient Modeling with Multi-modal Attentive Fusioncited by 4

Built on by

Not cited yet.