# Self-supervised 3D Human Mesh Recovery from Noisy Point Clouds arXiv: https://arxiv.org/abs/2107.07539 ยท HTML: https://mlsanity.com/paper/2107.07539/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Builds on 27 of its 52 references that are in the map. - First posted on arXiv 15 Jul 2021. ## Builds on (most cited first, 27 of 27) - [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017) - [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018) - [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016) - [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019) - [Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion](https://mlsanity.com/paper/2003.01456/) (2020) - [DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction](https://mlsanity.com/paper/1905.10711/) (2019) - [SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks](https://mlsanity.com/paper/2104.03313/) (2021) - [Panoptic Studio: A Massively Multiview System for Social Interaction Capture](https://mlsanity.com/paper/1612.03153/) (2016) - [3D-CODED : 3D Correspondences by Deep Deformation](https://mlsanity.com/paper/1806.05228/) (2018) - [Deep Functional Maps: Structured Prediction for Dense Shape Correspondence](https://mlsanity.com/paper/1704.08686/) (2017) - [Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction](https://mlsanity.com/paper/2007.11432/) (2020) - [LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration](https://mlsanity.com/paper/2010.12447/) (2020) - [LEAP: Learning Articulated Occupancy of People](https://mlsanity.com/paper/2104.06849/) (2021) - [DoubleFusion: Real-time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensor](https://mlsanity.com/paper/1804.06023/) (2018) - [Unsupervised Deep Learning for Structured Shape Matching](https://mlsanity.com/paper/1812.03794/) (2018) - [Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop](https://mlsanity.com/paper/2007.11110/) (2020) - [Deep Shells: Unsupervised Shape Correspondence with Optimal Transport](https://mlsanity.com/paper/2010.15261/) (2020) - [SCALE: Modeling Clothed Humans with a Surface Codec of Articulated Local Elements](https://mlsanity.com/paper/2104.07660/) (2021) - [Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration](https://mlsanity.com/paper/2104.08160/) (2021) - [Product Manifold Filter: Non-Rigid Shape Correspondence via Kernel Density Estimation in the Product Space](https://mlsanity.com/paper/1701.00669/) (2017) - [Cyclic Functional Mapping: Self-supervised correspondence between non-isometric deformable shapes](https://mlsanity.com/paper/1912.01249/) (2019) - [Dense Human Body Correspondences Using Convolutional Networks](https://mlsanity.com/paper/1511.05904/) (2015) - [FARM: Functional Automatic Registration Method for 3D Human Bodies](https://mlsanity.com/paper/1807.10517/) (2018) - [POSEFusion: Pose-guided Selective Fusion for Single-view Human Volumetric Capture](https://mlsanity.com/paper/2103.15331/) (2021) - [Recurrent Multi-view Alignment Network for Unsupervised Surface Registration](https://mlsanity.com/paper/2011.12104/) (2020) - [LBS Autoencoder: Self-supervised Fitting of Articulated Meshes to Point Clouds](https://mlsanity.com/paper/1904.10037/) (2019) - [Rigid and Articulated Point Registration with Expectation Conditional Maximization](https://mlsanity.com/paper/2012.05191/) (2020) ## Built on by (most cited first, 8 of 8) - [ARAH: Animatable Volume Rendering of Articulated Human SDFs](https://mlsanity.com/paper/2210.10036/) (2022) - [LiDAR-HMR: 3D Human Mesh Recovery from LiDAR](https://mlsanity.com/paper/2311.11971/) (2023) - [Geometric Prior Based Deep Human Point Cloud Geometry Compression](https://mlsanity.com/paper/2305.01309/) (2023) - [ReMP: Reusable Motion Prior for Multi-domain 3D Human Pose Estimation and Motion Inbetweening](https://mlsanity.com/paper/2411.09435/) (2024) - [Robust Human Registration with Body Part Segmentation on Noisy Point Clouds](https://mlsanity.com/paper/2504.03602/) (2025) - [AniGaussian: Animatable Gaussian Avatar with Pose-guided Deformation](https://mlsanity.com/paper/2502.19441/) (2025) - [Multi-modal 3D Pose and Shape Estimation with Computed Tomography](https://mlsanity.com/paper/2503.19405/) (2025) - [A Shape-Aware Total Body Photography System for In-focus Surface Coverage Optimization](https://mlsanity.com/paper/2505.16228/) (2025) ## Abstract (arXiv) This paper presents a novel self-supervised approach to reconstruct human shape and pose from noisy point cloud data. Relying on large amount of dataset with ground-truth annotations, recent learning-based approaches predict correspondences for every vertice on the point cloud; Chamfer distance is usually used to minimize the distance between a deformed template model and the input point cloud. However, Chamfer distance is quite sensitive to noise and outliers, thus could be unreliable to assign correspondences. To address these issues, we model the probability distribution of the input point cloud as generated from a parametric human model under a Gaussian Mixture Model. Instead of explicitly aligning correspondences, we treat the process of correspondence search as an implicit probabilistic association by updating the posterior probability of the template model given the input. A novel self-supervised loss is further derived which penalizes the discrepancy between the deformed template and the input point cloud conditioned on the posterior probability. Our approach is very flexible, which works with both complete point cloud and incomplete ones including even a single depth image as input. Compared to previous self-supervised methods, our method shows the capability to deal with substantial noise and outliers. Extensive experiments conducted on various public synthetic datasets as well as a very noisy real dataset (i.e. CMU Panoptic) demonstrate the superior performance of our approach over the state-of-the-art methods.