# OSSO: Obtaining Skeletal Shape from Outside arXiv: https://arxiv.org/abs/2204.10129 ยท HTML: https://mlsanity.com/paper/2204.10129/ ## Key facts - Cited by 25 papers in the map (data through 14 Oct 2026). - Builds on 13 of its 48 references that are in the map. - First posted on arXiv 21 Apr 2022. ## Builds on (most cited first, 13 of 13) - [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019) - [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019) - [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016) - [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017) - [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016) - [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018) - [3D Menagerie: Modeling the 3D shape and pose of animals](https://mlsanity.com/paper/1611.07700/) (2016) - [Learning to Estimate 3D Human Pose and Shape from a Single Color Image](https://mlsanity.com/paper/1805.04092/) (2018) - [Deep Learning-Based Human Pose Estimation: A Survey](https://mlsanity.com/paper/2012.13392/) (2020) - [Unite the People: Closing the Loop Between 3D and 2D Human Representations](https://mlsanity.com/paper/1701.02468/) (2017) - [AGORA: Avatars in Geography Optimized for Regression Analysis](https://mlsanity.com/paper/2104.14643/) (2021) - [STAR: Sparse Trained Articulated Human Body Regressor](https://mlsanity.com/paper/2008.08535/) (2020) - [BodyNet: Volumetric Inference of 3D Human Body Shapes](https://mlsanity.com/paper/1804.04875/) (2018) ## Built on by (most cited first, 25 of 25) - [TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videos](https://mlsanity.com/paper/2403.17346/) (2024) - [MedShapeNet -- A Large-Scale Dataset of 3D Medical Shapes for Computer Vision](https://mlsanity.com/paper/2308.16139/) (2023) - [From Skin to Skeleton: Towards Biomechanically Accurate 3D Digital Humans](https://mlsanity.com/paper/2509.06607/) (2025) - [Differentiable Biomechanics Unlocks Opportunities for Markerless Motion Capture](https://mlsanity.com/paper/2402.17192/) (2024) - [BOSS: Bones, Organs and Skin Shape Model](https://mlsanity.com/paper/2303.04923/) (2023) - [ATLAS: Decoupling Skeletal and Shape Parameters for Expressive Parametric Human Modeling](https://mlsanity.com/paper/2508.15767/) (2025) - [MHR: Momentum Human Rig](https://mlsanity.com/paper/2511.15586/) (2025) - [LOOC: Localizing Organs using Occupancy Networks and Body Surface Depth Images](https://mlsanity.com/paper/2406.12407/) (2024) - [Gaussian Control with Hierarchical Semantic Graphs in 3D Human Recovery](https://mlsanity.com/paper/2405.12477/) (2024) - [Automated CT Lung Cancer Screening Workflow using 3D Camera](https://mlsanity.com/paper/2309.15750/) (2023) - [3D Kinematics Estimation from Video with a Biomechanical Model and Synthetic Training Data](https://mlsanity.com/paper/2402.13172/) (2024) - [BFSM: 3D Bidirectional Face-Skull Morphable Model](https://mlsanity.com/paper/2509.24577/) (2025) - [Towards Next-Generation Healthcare: A Survey of Medical Embodied AI for Perception, Decision-Making, and Action](https://mlsanity.com/paper/2606.15647/) (2026) - [TailorMe: Self-Supervised Learning of an Anatomically Constrained Volumetric Human Shape Model](https://mlsanity.com/paper/2312.02173/) (2023) - [MaSkel: A Model for Human Whole-body X-rays Generation from Human Masking Images](https://mlsanity.com/paper/2404.09000/) (2024) - [Internal Organ Localization Using Depth Images](https://mlsanity.com/paper/2503.23468/) (2025) - [PIAvatar: Physically Interactive Avatars via Deformation Gradient Decoupling](https://mlsanity.com/paper/2606.21162/) (2026) - [Biomechanical 3D Body: Self-Supervised Distillation of Biomechanical Pose from a 3D Body Foundation Model](https://mlsanity.com/paper/2608.29928/) (2026) - [Development and Whole-Body Validation of Personalizable Female and Male Pedestrian SAFER Human Body Models](https://mlsanity.com/paper/2305.13918/) (2023) - [EA-RAS: Towards Efficient and Accurate End-to-End Reconstruction of Anatomical Skeleton](https://mlsanity.com/paper/2409.01555/) (2024) - [BiomechGPT: Extending Motion-Language Models to Clinical Motion Understanding](https://mlsanity.com/paper/2505.18465/) (2025) - [Depth to Anatomy: Organ Localization from Depth Images for Automated Patient Table Positioning in Radiology Workflow](https://mlsanity.com/paper/2601.18260/) (2026) - [Patient4D: Temporally Consistent Patient Body Mesh Recovery from Monocular Operating Room Video](https://mlsanity.com/paper/2603.17178/) (2026) - [BioHuman: Learning Biomechanical Human Representations from Video](https://mlsanity.com/paper/2605.14772/) (2026) - [SOMA: From Surface Observations to Muscle Anatomy](https://mlsanity.com/paper/2606.09246/) (2026) ## Abstract (arXiv) We address the problem of inferring the anatomic skeleton of a person, in an arbitrary pose, from the 3D surface of the body; i.e. we predict the inside (bones) from the outside (skin). This has many applications in medicine and biomechanics. Existing state-of-the-art biomechanical skeletons are detailed but do not easily generalize to new subjects. Additionally, computer vision and graphics methods that predict skeletons are typically heuristic, not learned from data, do not leverage the full 3D body surface, and are not validated against ground truth. To our knowledge, our system, called OSSO (Obtaining Skeletal Shape from Outside), is the first to learn the mapping from the 3D body surface to the internal skeleton from real data. We do so using 1000 male and 1000 female dual-energy X-ray absorptiometry (DXA) scans. To these, we fit a parametric 3D body shape model (STAR) to capture the body surface and a novel part-based 3D skeleton model to capture the bones. This provides inside/outside training pairs. We model the statistical variation of full skeletons using PCA in a pose-normalized space. We then train a regressor from body shape parameters to skeleton shape parameters and refine the skeleton to satisfy constraints on physical plausibility. Given an arbitrary 3D body shape and pose, OSSO predicts a realistic skeleton inside. In contrast to previous work, we evaluate the accuracy of the skeleton shape quantitatively on held-out DXA scans, outperforming the state-of-the-art. We also show 3D skeleton prediction from varied and challenging 3D bodies. The code to infer a skeleton from a body shape is available for research at https://osso.is.tue.mpg.de/, and the dataset of paired outer surface (skin) and skeleton (bone) meshes is available as a Biobank Returned Dataset. This research has been conducted using the UK Biobank Resource.