# On Self-Contact and Human Pose arXiv: https://arxiv.org/abs/2104.03176 ยท HTML: https://mlsanity.com/paper/2104.03176/ ## Key facts - Cited by 89 papers in the map (data through 14 Oct 2026). - Builds on 17 of its 50 references that are in the map. - First posted on arXiv 7 Apr 2021. ## Builds on (most cited first, 17 of 17) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019) - [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019) - [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017) - [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016) - [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019) - [Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision](https://mlsanity.com/paper/1611.09813/) (2016) - [Resolving 3D Human Pose Ambiguities with 3D Scene Constraints](https://mlsanity.com/paper/1908.06963/) (2019) - [Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies](https://mlsanity.com/paper/1801.01615/) (2018) - [Monocular Total Capture: Posing Face, Body, and Hands in the Wild](https://mlsanity.com/paper/1812.01598/) (2018) - [Ordinal Depth Supervision for 3D Human Pose Estimation](https://mlsanity.com/paper/1805.04095/) (2018) - [XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera](https://mlsanity.com/paper/1907.00837/) (2019) - [MonoClothCap: Towards Temporally Coherent Clothing Capture from Monocular RGB Video](https://mlsanity.com/paper/2009.10711/) (2020) - [Estimating 3D Motion and Forces of Person-Object Interactions from Monocular Video](https://mlsanity.com/paper/1904.02683/) (2019) - [Learning Complex 3D Human Self-Contact](https://mlsanity.com/paper/2012.10366/) (2020) - [Detecting Hands and Recognizing Physical Contact in the Wild](https://mlsanity.com/paper/2010.09676/) (2020) - [A survey on Deep Learning Advances on Different 3D Data Representations](https://mlsanity.com/paper/1808.01462/) (2018) ## Built on by (most cited first, 30 of 89) - [BEHAVE: Dataset and Method for Tracking Human Object Interactions](https://mlsanity.com/paper/2204.06950/) (2022) - [CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimation](https://mlsanity.com/paper/2208.00571/) (2022) - [BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion](https://mlsanity.com/paper/2306.16940/) (2023) - [Capturing and Inferring Dense Full-Body Human-Scene Contact](https://mlsanity.com/paper/2206.09553/) (2022) - [SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimation](https://mlsanity.com/paper/2309.17448/) (2023) - [Probabilistic Modeling for Human Mesh Recovery](https://mlsanity.com/paper/2108.11944/) (2021) - [GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras](https://mlsanity.com/paper/2112.01524/) (2021) - [PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images](https://mlsanity.com/paper/2207.06400/) (2022) - [Putting People in their Place: Monocular Regression of 3D People in Depth](https://mlsanity.com/paper/2112.08274/) (2021) - [Collaborative Regression of Expressive Bodies using Moderation](https://mlsanity.com/paper/2105.05301/) (2021) - [SPEC: Seeing People in the Wild with an Estimated Camera](https://mlsanity.com/paper/2110.00620/) (2021) - [One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformer](https://mlsanity.com/paper/2303.16160/) (2023) - [HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling](https://mlsanity.com/paper/2204.13686/) (2022) - [Recovering 3D Human Mesh from Monocular Images: A Survey](https://mlsanity.com/paper/2203.01923/) (2022) - [3D Human Pose Estimation via Intuitive Physics](https://mlsanity.com/paper/2303.18246/) (2023) - [CHORE: Contact, Human and Object REconstruction from a single RGB image](https://mlsanity.com/paper/2204.02445/) (2022) - [Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applications](https://mlsanity.com/paper/2208.03826/) (2022) - [Human-Art: A Versatile Human-Centric Dataset Bridging Natural and Artificial Scenes](https://mlsanity.com/paper/2303.02760/) (2023) - [Learning to Estimate Robust 3D Human Mesh from In-the-Wild Crowded Scenes](https://mlsanity.com/paper/2104.07300/) (2021) - [DECO: Dense Estimation of 3D Human-Scene Contact In The Wild](https://mlsanity.com/paper/2309.15273/) (2023) - [Accurate 3D Body Shape Regression using Metric and Semantic Attributes](https://mlsanity.com/paper/2206.07036/) (2022) - [ReFit: Recurrent Fitting Network for 3D Human Recovery](https://mlsanity.com/paper/2308.11184/) (2023) - [Generative Proxemics: A Prior for 3D Social Interaction from Images](https://mlsanity.com/paper/2306.09337/) (2023) - [Zolly: Zoom Focal Length Correctly for Perspective-Distorted Human Mesh Reconstruction](https://mlsanity.com/paper/2303.13796/) (2023) - [PACE: Human and Camera Motion Estimation from in-the-wild Videos](https://mlsanity.com/paper/2310.13768/) (2023) - [Probabilistic Human Mesh Recovery in 3D Scenes from Egocentric Views](https://mlsanity.com/paper/2304.06024/) (2023) - [Contact-Aware Retargeting of Skinned Motion](https://mlsanity.com/paper/2109.07431/) (2021) - [SMPLest-X: Ultimate Scaling for Expressive Human Pose and Shape Estimation](https://mlsanity.com/paper/2501.09782/) (2025) - [HSPACE: Synthetic Parametric Humans Animated in Complex Environments](https://mlsanity.com/paper/2112.12867/) (2021) - [HULC: 3D Human Motion Capture with Pose Manifold Sampling and Dense Contact Guidance](https://mlsanity.com/paper/2205.05677/) (2022) ## Abstract (arXiv) People touch their face 23 times an hour, they cross their arms and legs, put their hands on their hips, etc. While many images of people contain some form of self-contact, current 3D human pose and shape (HPS) regression methods typically fail to estimate this contact. To address this, we develop new datasets and methods that significantly improve human pose estimation with self-contact. First, we create a dataset of 3D Contact Poses (3DCP) containing SMPL-X bodies fit to 3D scans as well as poses from AMASS, which we refine to ensure good contact. Second, we leverage this to create the Mimic-The-Pose (MTP) dataset of images, collected via Amazon Mechanical Turk, containing people mimicking the 3DCP poses with selfcontact. Third, we develop a novel HPS optimization method, SMPLify-XMC, that includes contact constraints and uses the known 3DCP body pose during fitting to create near ground-truth poses for MTP images. Fourth, for more image variety, we label a dataset of in-the-wild images with Discrete Self-Contact (DSC) information and use another new optimization method, SMPLify-DC, that exploits discrete contacts during pose optimization. Finally, we use our datasets during SPIN training to learn a new 3D human pose regressor, called TUCH (Towards Understanding Contact in Humans). We show that the new self-contact training data significantly improves 3D human pose estimates on withheld test data and existing datasets like 3DPW. Not only does our method improve results for self-contact poses, but it also improves accuracy for non-contact poses. The code and data are available for research purposes at https://tuch.is.tue.mpg.de.