# DiffAvatar: Simulation-Ready Garment Optimization with Differentiable Simulation

arXiv: https://arxiv.org/abs/2311.12194 · HTML: https://mlsanity.com/paper/2311.12194/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 68 references that are in the map.
- First posted on arXiv 20 Nov 2023.

## Builds on (most cited first, 30 of 34)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Wonder3D: Single Image to 3D using Cross-Domain Diffusion](https://mlsanity.com/paper/2310.15008/) (2023)
- [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020)
- [Neural-GIF: Neural Generalized Implicit Functions for Animating People in Clothing](https://mlsanity.com/paper/2108.08807/) (2021)
- [The Power of Points for Modeling Humans in Clothing](https://mlsanity.com/paper/2109.01137/) (2021)
- [SNUG: Self-Supervised Neural Dynamic Garments](https://mlsanity.com/paper/2204.02219/) (2022)
- [Self-Supervised Collision Handling via Generative 3D Garment Models for Virtual Try-On](https://mlsanity.com/paper/2105.06462/) (2021)
- [HOOD: Hierarchical Graphs for Generalized Modelling of Clothing Dynamics](https://mlsanity.com/paper/2212.07242/) (2022)
- [DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contact](https://mlsanity.com/paper/2106.05306/) (2021)
- [DiffPD: Differentiable Projective Dynamics](https://mlsanity.com/paper/2101.05917/) (2021)
- [DrapeNet: Garment Generation and Self-Supervised Draping](https://mlsanity.com/paper/2211.11277/) (2022)
- [Neural Cloth Simulation](https://mlsanity.com/paper/2212.11220/) (2022)
- [Virtual Elastic Objects](https://mlsanity.com/paper/2201.04623/) (2022)
- [Towards Garment Sewing Pattern Reconstruction from a Single Image](https://mlsanity.com/paper/2311.04218/) (2023)
- [ISP: Multi-Layered Garment Draping with Implicit Sewing Patterns](https://mlsanity.com/paper/2305.14100/) (2023)
- [Deep Physics-aware Inference of Cloth Deformation for Monocular Human Performance Capture](https://mlsanity.com/paper/2011.12866/) (2020)
- [NeuWigs: A Neural Dynamic Model for Volumetric Hair Capture and Animation](https://mlsanity.com/paper/2212.00613/) (2022)
- [HVH: Learning a Hybrid Neural Volumetric Representation for Dynamic Hair Performance Capture](https://mlsanity.com/paper/2112.06904/) (2021)
- [REC-MV: REconstructing 3D Dynamic Cloth from Monocular Videos](https://mlsanity.com/paper/2305.14236/) (2023)
- [GarmentCode: Programming Parametric Sewing Patterns](https://mlsanity.com/paper/2306.03642/) (2023)
- [DIG: Draping Implicit Garment over the Human Body](https://mlsanity.com/paper/2209.10845/) (2022)
- [TightCap: 3D Human Shape Capture with Clothing Tightness Field](https://mlsanity.com/paper/1904.02601/) (2019)
- [Computational Pattern Making from 3D Garment Models](https://mlsanity.com/paper/2202.10272/) (2022)
- [PERGAMO: Personalized 3D Garments from Monocular Video](https://mlsanity.com/paper/2210.15040/) (2022)
- [NSF: Neural Surface Fields for Human Modeling from Monocular Depth](https://mlsanity.com/paper/2308.14847/) (2023)
- [CaPhy: Capturing Physical Properties for Animatable Human Avatars](https://mlsanity.com/paper/2308.05925/) (2023)
- [Structure-Preserving 3D Garment Modeling with Neural Sewing Machines](https://mlsanity.com/paper/2211.06701/) (2022)
- [How Will It Drape Like? Capturing Fabric Mechanics from Depth Images](https://mlsanity.com/paper/2304.06704/) (2023)
- [IMP: Instance Mask Projection for High Accuracy Semantic Segmentation of Things](https://mlsanity.com/paper/1906.06597/) (2019)

## Built on by (most cited first, 30 of 41)

- [PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual Observations](https://mlsanity.com/paper/2404.04421/) (2024)
- [DressCode: Autoregressively Sewing and Generating Garments from Text Guidance](https://mlsanity.com/paper/2401.16465/) (2024)
- [Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Models](https://mlsanity.com/paper/2406.08475/) (2024)
- [Garment3DGen: 3D Garment Stylization and Texture Generation](https://mlsanity.com/paper/2403.18816/) (2024)
- [Gaussian Garments: Reconstructing Simulation-Ready Clothing with Photorealistic Appearance from Multi-View Video](https://mlsanity.com/paper/2409.08189/) (2024)
- [LayGA: Layered Gaussian Avatars for Animatable Clothing Transfer](https://mlsanity.com/paper/2405.07319/) (2024)
- [Atlas3D: Physically Constrained Self-Supporting Text-to-3D for Simulation and Fabrication](https://mlsanity.com/paper/2405.18515/) (2024)
- [Dress-1-to-3: Single Image to Simulation-Ready 3D Outfit with Diffusion Prior and Differentiable Physics](https://mlsanity.com/paper/2502.03449/) (2025)
- [GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Details](https://mlsanity.com/paper/2405.12420/) (2024)
- [Design2GarmentCode: Turning Design Concepts to Tangible Garments Through Program Synthesis](https://mlsanity.com/paper/2412.08603/) (2024)
- [Gen-3Diffusion: Realistic Image-to-3D Generation via 2D & 3D Diffusion Synergy](https://mlsanity.com/paper/2412.06698/) (2024)
- [NeuralFluid: Neural Fluidic System Design and Control with Differentiable Simulation](https://mlsanity.com/paper/2405.14903/) (2024)
- [ClotheDreamer: Text-Guided Garment Generation with 3D Gaussians](https://mlsanity.com/paper/2406.16815/) (2024)
- [MPMAvatar: Learning 3D Gaussian Avatars with Accurate and Robust Physics-Based Dynamics](https://mlsanity.com/paper/2510.01619/) (2025)
- [ParticleGS: Learning Neural Gaussian Particle Dynamics from Videos for Prior-free Physical Motion Extrapolation](https://mlsanity.com/paper/2505.20270/) (2025)
- [GarmageNet: A Multimodal Generative Framework for Sewing Pattern Design and Generic Garment Modeling](https://mlsanity.com/paper/2504.01483/) (2025)
- [WaSt-3D: Wasserstein-2 Distance for Scene-to-Scene Stylization on 3D Gaussians](https://mlsanity.com/paper/2409.17917/) (2024)
- [3DV-TON: Textured 3D-Guided Consistent Video Try-on via Diffusion Models](https://mlsanity.com/paper/2504.17414/) (2025)
- [GarmentX: Autoregressive Parametric Representations for High-Fidelity 3D Garment Generation](https://mlsanity.com/paper/2504.20409/) (2025)
- [Image2Garment: Simulation-ready Garment Generation from a Single Image](https://mlsanity.com/paper/2601.09658/) (2026)
- [NeuROK: Generative 4D Neural Object Kinematics](https://mlsanity.com/paper/2605.30347/) (2026)
- [CRAFT: Designing Creative and Functional 3D Objects](https://mlsanity.com/paper/2412.03889/) (2024)
- [SpringTime: Learning Simulatable Models of Cloth with Spatially-varying Constitutive Properties](https://mlsanity.com/paper/2507.21288/) (2025)
- [NGD: Neural Gradient Based Deformation for Monocular Garment Reconstruction](https://mlsanity.com/paper/2508.17712/) (2025)
- [DressWild: Feed-Forward Pose-Agnostic Garment Sewing Pattern Generation from In-the-Wild Images](https://mlsanity.com/paper/2602.16502/) (2026)
- [Gaussian Wardrobe: Compositional 3D Gaussian Avatars for Free-Form Virtual Try-On](https://mlsanity.com/paper/2603.04290/) (2026)
- [DAMA: Disentangled Body-Anchored Gaussians for Controllable Multi-Layered Avatars](https://mlsanity.com/paper/2605.21001/) (2026)
- [FabricDiffusion: High-Fidelity Texture Transfer for 3D Garments Generation from In-The-Wild Clothing Images](https://mlsanity.com/paper/2410.01801/) (2024)
- [FashionR2R: Texture-preserving Rendered-to-Real Image Translation with Diffusion Models](https://mlsanity.com/paper/2410.14429/) (2024)
- [Neural Modular Physics for Elastic Simulation](https://mlsanity.com/paper/2512.15083/) (2025)

## Abstract (arXiv)

The realism of digital avatars is crucial in enabling telepresence applications with self-expression and customization. While physical simulations can produce realistic motions for clothed humans, they require high-quality garment assets with associated physical parameters for cloth simulations. However, manually creating these assets and calibrating their parameters is labor-intensive and requires specialized expertise. Current methods focus on reconstructing geometry, but don't generate complete assets for physics-based applications. To address this gap, we propose papername,~a novel approach that performs body and garment co-optimization using differentiable simulation. By integrating physical simulation into the optimization loop and accounting for the complex nonlinear behavior of cloth and its intricate interaction with the body, our framework recovers body and garment geometry and extracts important material parameters in a physically plausible way. Our experiments demonstrate that our approach generates realistic clothing and body shape suitable for downstream applications. We provide additional insights and results on our webpage: https://people.csail.mit.edu/liyifei/publication/diffavatar/