# AniDress: Animatable Loose-Dressed Avatar from Sparse Views Using Garment Rigging Model

arXiv: https://arxiv.org/abs/2401.15348 · HTML: https://mlsanity.com/paper/2401.15348/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 77 references that are in the map.
- First posted on arXiv 27 Jan 2024.

## Builds on (most cited first, 30 of 55)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video](https://mlsanity.com/paper/2201.04127/) (2022)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies](https://mlsanity.com/paper/2105.02872/) (2021)
- [ICON: Implicit Clothed humans Obtained from Normals](https://mlsanity.com/paper/2112.09127/) (2021)
- [SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes](https://mlsanity.com/paper/2104.03953/) (2021)
- [ECON: Explicit Clothed humans Optimized via Normal integration](https://mlsanity.com/paper/2212.07422/) (2022)
- [Neural Articulated Radiance Field](https://mlsanity.com/paper/2104.03110/) (2021)
- [SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks](https://mlsanity.com/paper/2104.03313/) (2021)
- [Self-Correction for Human Parsing](https://mlsanity.com/paper/1910.09777/) (2019)
- [TAVA: Template-free Animatable Volumetric Actors](https://mlsanity.com/paper/2206.08929/) (2022)
- [DeepCap: Monocular Human Performance Capture Using Weak Supervision](https://mlsanity.com/paper/2003.08325/) (2020)
- [HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion](https://mlsanity.com/paper/2305.06356/) (2023)
- [SMPLicit: Topology-aware Generative Model for Clothed People](https://mlsanity.com/paper/2103.06871/) (2021)
- [TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style](https://mlsanity.com/paper/2003.04583/) (2020)
- [ARAH: Animatable Volume Rendering of Articulated Human SDFs](https://mlsanity.com/paper/2210.10036/) (2022)
- [Real-time Deep Dynamic Characters](https://mlsanity.com/paper/2105.01794/) (2021)
- [BCNet: Learning Body and Cloth Shape from A Single Image](https://mlsanity.com/paper/2004.00214/) (2020)
- [Structured Local Radiance Fields for Human Avatar Modeling](https://mlsanity.com/paper/2203.14478/) (2022)
- [LEAP: Learning Articulated Occupancy of People](https://mlsanity.com/paper/2104.06849/) (2021)
- [Deep Fashion3D: A Dataset and Benchmark for 3D Garment Reconstruction from Single Images](https://mlsanity.com/paper/2003.12753/) (2020)
- [Neural-GIF: Neural Generalized Implicit Functions for Animating People in Clothing](https://mlsanity.com/paper/2108.08807/) (2021)
- [DeepWrinkles: Accurate and Realistic Clothing Modeling](https://mlsanity.com/paper/1808.03417/) (2018)

## Built on by (most cited first, 3 of 3)

- [PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual Observations](https://mlsanity.com/paper/2404.04421/) (2024)
- [Multimodal Latent Diffusion Model for Complex Sewing Pattern Generation](https://mlsanity.com/paper/2412.14453/) (2024)
- [A Comparative Study of Garment Draping Techniques](https://mlsanity.com/paper/2405.11056/) (2024)

## Abstract (arXiv)

Recent communities have seen significant progress in building photo-realistic animatable avatars from sparse multi-view videos. However, current workflows struggle to render realistic garment dynamics for loose-fitting characters as they predominantly rely on naked body models for human modeling while leaving the garment part un-modeled. This is mainly due to that the deformations yielded by loose garments are highly non-rigid, and capturing such deformations often requires dense views as supervision. In this paper, we introduce AniDress, a novel method for generating animatable human avatars in loose clothes using very sparse multi-view videos (4-8 in our setting). To allow the capturing and appearance learning of loose garments in such a situation, we employ a virtual bone-based garment rigging model obtained from physics-based simulation data. Such a model allows us to capture and render complex garment dynamics through a set of low-dimensional bone transformations. Technically, we develop a novel method for estimating temporal coherent garment dynamics from a sparse multi-view video. To build a realistic rendering for unseen garment status using coarse estimations, a pose-driven deformable neural radiance field conditioned on both body and garment motions is introduced, providing explicit control of both parts. At test time, the new garment poses can be captured from unseen situations, derived from a physics-based or neural network-based simulator to drive unseen garment dynamics. To evaluate our approach, we create a multi-view dataset that captures loose-dressed performers with diverse motions. Experiments show that our method is able to render natural garment dynamics that deviate highly from the body and generalize well to both unseen views and poses, surpassing the performance of existing methods. The code and data will be publicly available.