# PhysXNet: A Customizable Approach for LearningCloth Dynamics on Dressed People

arXiv: https://arxiv.org/abs/2111.07195 · HTML: https://mlsanity.com/paper/2111.07195/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 40 references that are in the map.
- First posted on arXiv 13 Nov 2021.

## Builds on (most cited first, 23 of 23)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Learning from Synthetic Humans](https://mlsanity.com/paper/1701.01370/) (2017)
- [Multi-Garment Net: Learning to Dress 3D People from Images](https://mlsanity.com/paper/1908.06903/) (2019)
- [DeepCap: Monocular Human Performance Capture Using Weak Supervision](https://mlsanity.com/paper/2003.08325/) (2020)
- [SMPLicit: Topology-aware Generative Model for Clothed People](https://mlsanity.com/paper/2103.06871/) (2021)
- [TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style](https://mlsanity.com/paper/2003.04583/) (2020)
- [CLOTH3D: Clothed 3D Humans](https://mlsanity.com/paper/1912.02792/) (2019)
- [DeepWrinkles: Accurate and Realistic Clothing Modeling](https://mlsanity.com/paper/1808.03417/) (2018)
- [Learning-Based Animation of Clothing for Virtual Try-On](https://mlsanity.com/paper/1903.07190/) (2019)
- [SIZER: A Dataset and Model for Parsing 3D Clothing and Learning Size Sensitive 3D Clothing](https://mlsanity.com/paper/2007.11610/) (2020)
- [Self-Supervised Collision Handling via Generative 3D Garment Models for Virtual Try-On](https://mlsanity.com/paper/2105.06462/) (2021)
- [GarNet: A Two-Stream Network for Fast and Accurate 3D Cloth Draping](https://mlsanity.com/paper/1811.10983/) (2018)
- [3DPeople: Modeling the Geometry of Dressed Humans](https://mlsanity.com/paper/1904.04571/) (2019)
- [Learning a Shared Shape Space for Multimodal Garment Design](https://mlsanity.com/paper/1806.11335/) (2018)
- [Fully Convolutional Graph Neural Networks for Parametric Virtual Try-On](https://mlsanity.com/paper/2009.04592/) (2020)
- [SimulCap : Single-View Human Performance Capture with Cloth Simulation](https://mlsanity.com/paper/1903.06323/) (2019)
- [Deep Physics-aware Inference of Cloth Deformation for Monocular Human Performance Capture](https://mlsanity.com/paper/2011.12866/) (2020)
- [A Pixel-Based Framework for Data-Driven Clothing](https://mlsanity.com/paper/1812.01677/) (2018)
- [Deep Detail Enhancement for Any Garment](https://mlsanity.com/paper/2008.04367/) (2020)
- [Coercing Machine Learning to Output Physically Accurate Results](https://mlsanity.com/paper/1910.09671/) (2019)
- [AVATAR: Blender add-on for fast creation of 3D human models](https://mlsanity.com/paper/2103.14507/) (2021)

## Built on by (most cited first, 1 of 1)

- [A Generative Multi-Resolution Pyramid and Normal-Conditioning 3D Cloth Draping](https://mlsanity.com/paper/2311.02700/) (2023)

## Abstract (arXiv)

We introduce PhysXNet, a learning-based approach to predict the dynamics of deformable clothes given 3D skeleton motion sequences of humans wearing these clothes. The proposed model is adaptable to a large variety of garments and changing topologies, without need of being retrained. Such simulations are typically carried out by physics engines that require manual human expertise and are subjectto computationally intensive computations. PhysXNet, by contrast, is a fully differentiable deep network that at inference is able to estimate the geometry of dense cloth meshes in a matter of milliseconds, and thus, can be readily deployed as a layer of a larger deep learning architecture. This efficiency is achieved thanks to the specific parameterization of the clothes we consider, based on 3D UV maps encoding spatial garment displacements. The problem is then formulated as a mapping between the human kinematics space (represented also by 3D UV maps of the undressed body mesh) into the clothes displacement UV maps, which we learn using a conditional GAN with a discriminator that enforces feasible deformations. We train simultaneously our model for three garment templates, tops, bottoms and dresses for which we simulate deformations under 50 different human actions. Nevertheless, the UV map representation we consider allows encapsulating many different cloth topologies, and at test we can simulate garments even if we did not specifically train for them. A thorough evaluation demonstrates that PhysXNet delivers cloth deformations very close to those computed with the physical engine, opening the door to be effectively integrated within deeplearning pipelines.