# DiT-Garment: Garment Dynamics with Diffusion Transformers

arXiv: https://arxiv.org/abs/2609.18510 · HTML: https://mlsanity.com/paper/2609.18510/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 36 of its 56 references that are in the map.
- First posted on arXiv 16 Sep 2026.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 36)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [SMPLicit: Topology-aware Generative Model for Clothed People](https://mlsanity.com/paper/2103.06871/) (2021)
- [TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style](https://mlsanity.com/paper/2003.04583/) (2020)
- [CLOTH3D: Clothed 3D Humans](https://mlsanity.com/paper/1912.02792/) (2019)
- [DeepWrinkles: Accurate and Realistic Clothing Modeling](https://mlsanity.com/paper/1808.03417/) (2018)
- [The Power of Points for Modeling Humans in Clothing](https://mlsanity.com/paper/2109.01137/) (2021)
- [SNUG: Self-Supervised Neural Dynamic Garments](https://mlsanity.com/paper/2204.02219/) (2022)
- [Self-Supervised Collision Handling via Generative 3D Garment Models for Virtual Try-On](https://mlsanity.com/paper/2105.06462/) (2021)
- [HOOD: Hierarchical Graphs for Generalized Modelling of Clothing Dynamics](https://mlsanity.com/paper/2212.07242/) (2022)
- [SCALE: Modeling Clothed Humans with a Surface Codec of Articulated Local Elements](https://mlsanity.com/paper/2104.07660/) (2021)
- [PBNS: Physically Based Neural Simulator for Unsupervised Garment Pose Space Deformation](https://mlsanity.com/paper/2012.11310/) (2020)
- [4D-DRESS: A 4D Dataset of Real-world Human Clothing with Semantic Annotations](https://mlsanity.com/paper/2404.18630/) (2024)
- [Learning Implicit Templates for Point-Based Clothed Human Modeling](https://mlsanity.com/paper/2207.06955/) (2022)
- [DrapeNet: Garment Generation and Self-Supervised Draping](https://mlsanity.com/paper/2211.11277/) (2022)
- [Point-Based Modeling of Human Clothing](https://mlsanity.com/paper/2104.08230/) (2021)
- [Dynamic Point Fields](https://mlsanity.com/paper/2304.02626/) (2023)
- [Neural Point-based Shape Modeling of Humans in Challenging Clothing](https://mlsanity.com/paper/2209.06814/) (2022)
- [GarmentCodeData: A Dataset of 3D Made-to-Measure Garments With Sewing Patterns](https://mlsanity.com/paper/2405.17609/) (2024)
- [GarmentCode: Programming Parametric Sewing Patterns](https://mlsanity.com/paper/2306.03642/) (2023)
- [DIG: Draping Implicit Garment over the Human Body](https://mlsanity.com/paper/2209.10845/) (2022)
- [Garment Recovery with Shape and Deformation Priors](https://mlsanity.com/paper/2311.10356/) (2023)
- [ContourCraft: Learning to Resolve Intersections in Neural Multi-Garment Simulations](https://mlsanity.com/paper/2405.09522/) (2024)
- [Motion Guided Deep Dynamic 3D Garments](https://mlsanity.com/paper/2209.11449/) (2022)
- [4DHumanOutfit: a multi-subject 4D dataset of human motion sequences in varying outfits exhibiting large displacements](https://mlsanity.com/paper/2306.07399/) (2023)
- [GAPS: Geometry-Aware, Physics-Based, Self-Supervised Neural Garment Draping](https://mlsanity.com/paper/2312.01490/) (2023)
- [Reconstruction of Manipulated Garment with Guided Deformation Prior](https://mlsanity.com/paper/2405.10934/) (2024)

## Abstract (arXiv)

We present DiT-Garment to model dynamic 3D clothing over human body models in arbitrary motion. Unlike existing methods, DiT-Garment can animate garments with unseen designs and physical materials, while allowing for direct inference of deformations for any target pose. To achieve this, we leverage a 2D diffusion transformer architecture to learn 3D deformations in a 2D UV-space. As the result is non-deterministic, our generative model learns the distribution of possible outcomes. The template garment is represented as a 3D triangle mesh spatially aligned with a 3D human body model in a standardized pose. To work with different garment designs without the need of a common template or complex graph convolution operations, the diffusion transformer is conditioned on a 3D position map of the template, represented in UV-space, which allows to implicitly learn a deformation of the 3D space around the body in standard pose. Further conditioning on body motion and physical parameters allows to physically ground the model. We quantitatively and qualitatively evaluate DiT-Garment on both synthetic and real data. While only trained on synthetic simulations of automatically generated cloth designs, our method generalizes to captured and artist-made garment designs. Code and data are available for research purposes at https://dumoulina.github.io/dit-garment/.