# Deep Deformation Detail Synthesis for Thin Shell Models

arXiv: https://arxiv.org/abs/2102.11541 · HTML: https://mlsanity.com/paper/2102.11541/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 89 references that are in the map.
- First posted on arXiv 23 Feb 2021.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [Geometric deep learning on graphs and manifolds using mixture model CNNs](https://mlsanity.com/paper/1611.08402/) (2016)
- [Video Action Transformer Network](https://mlsanity.com/paper/1812.02707/) (2018)
- [DiffusionNet: Discretization Agnostic Learning on Surfaces](https://mlsanity.com/paper/2012.00888/) (2020)
- [SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels](https://mlsanity.com/paper/1711.08920/) (2017)
- [TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style](https://mlsanity.com/paper/2003.04583/) (2020)
- [Learning Rich Features from RGB-D Images for Object Detection and Segmentation](https://mlsanity.com/paper/1407.5736/) (2014)
- [Learning shape correspondence with anisotropic convolutional neural networks](https://mlsanity.com/paper/1605.06437/) (2016)
- [DeepWrinkles: Accurate and Realistic Clothing Modeling](https://mlsanity.com/paper/1808.03417/) (2018)
- [Learning-Based Animation of Clothing for Virtual Try-On](https://mlsanity.com/paper/1903.07190/) (2019)
- [Variational Autoencoders for Deforming 3D Mesh Models](https://mlsanity.com/paper/1709.04307/) (2017)
- [MeshCNN: A Network with an Edge](https://mlsanity.com/paper/1809.05910/) (2018)
- [GarNet: A Two-Stream Network for Fast and Accurate 3D Cloth Draping](https://mlsanity.com/paper/1811.10983/) (2018)
- [PBNS: Physically Based Neural Simulator for Unsupervised Garment Pose Space Deformation](https://mlsanity.com/paper/2012.11310/) (2020)
- [Disentangled Representation Learning for 3D Face Shape](https://mlsanity.com/paper/1902.09887/) (2019)
- [SurfNet: Generating 3D shape surfaces using deep residual networks](https://mlsanity.com/paper/1703.04079/) (2017)
- [3D Face Reconstruction with Geometry Details from a Single Image](https://mlsanity.com/paper/1702.05619/) (2017)
- [GarNet++: Improving Fast and Accurate Static3D Cloth Draping by Curvature Loss](https://mlsanity.com/paper/2007.10867/) (2020)
- [Deep Detail Enhancement for Any Garment](https://mlsanity.com/paper/2008.04367/) (2020)
- [Sparse Data Driven Mesh Deformation](https://mlsanity.com/paper/1709.01250/) (2017)
- [Disentangled Human Body Embedding Based on Deep Hierarchical Neural Network](https://mlsanity.com/paper/1905.05622/) (2019)

## Built on by (most cited first, 5 of 5)

- [Predicting Loose-Fitting Garment Deformations Using Bone-Driven Motion Networks](https://mlsanity.com/paper/2205.01355/) (2022)
- [GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Details](https://mlsanity.com/paper/2405.12420/) (2024)
- [N-Cloth: Predicting 3D Cloth Deformation with Mesh-Based Networks](https://mlsanity.com/paper/2112.06397/) (2021)
- [Smart Fashion: A Review of AI Applications in the Fashion & Apparel Industry](https://mlsanity.com/paper/2111.00905/) (2021)
- [Neural Garment Dynamic Super-Resolution](https://mlsanity.com/paper/2412.06285/) (2024)

## Abstract (arXiv)

In physics-based cloth animation, rich folds and detailed wrinkles are achieved at the cost of expensive computational resources and huge labor tuning. Data-driven techniques make efforts to reduce the computation significantly by a database. One type of methods relies on human poses to synthesize fitted garments which cannot be applied to general cloth. Another type of methods adds details to the coarse meshes without such restrictions. However, existing works usually utilize coordinate-based representations which cannot cope with large-scale deformation, and requires dense vertex correspondences between coarse and fine meshes. Moreover, as such methods only add details, they require coarse meshes to be close to fine meshes, which can be either impossible, or require unrealistic constraints when generating fine meshes. To address these challenges, we develop a temporally and spatially as-consistent-as-possible deformation representation (named TS-ACAP) and a DeformTransformer network to learn the mapping from low-resolution meshes to detailed ones. This TS-ACAP representation is designed to ensure both spatial and temporal consistency for sequential large-scale deformations from cloth animations. With this representation, our DeformTransformer network first utilizes two mesh-based encoders to extract the coarse and fine features, respectively. To transduct the coarse features to the fine ones, we leverage the Transformer network that consists of frame-level attention mechanisms to ensure temporal coherence of the prediction. Experimental results show that our method is able to produce reliable and realistic animations in various datasets at high frame rates: 10 ~ 35 times faster than physics-based simulation, with superior detail synthesis abilities than existing methods.