# Drivable Avatar Clothing: Faithful Full-Body Telepresence with Dynamic Clothing Driven by Sparse RGB-D Input

arXiv: https://arxiv.org/abs/2310.05917 · HTML: https://mlsanity.com/paper/2310.05917/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 63 of its 85 references that are in the map.
- First posted on arXiv 9 Oct 2023.

## Builds on (most cited first, 30 of 63)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020)
- [Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies](https://mlsanity.com/paper/2105.02872/) (2021)
- [ICON: Implicit Clothed humans Obtained from Normals](https://mlsanity.com/paper/2112.09127/) (2021)
- [Mixture of Volumetric Primitives for Efficient Neural Rendering](https://mlsanity.com/paper/2103.01954/) (2021)
- [Deferred Neural Rendering: Image Synthesis using Neural Textures](https://mlsanity.com/paper/1904.12356/) (2019)
- [Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensors](https://mlsanity.com/paper/2105.01859/) (2021)
- [A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose](https://mlsanity.com/paper/2102.06199/) (2021)
- [Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Rendering](https://mlsanity.com/paper/2211.11610/) (2022)
- [ECON: Explicit Clothed humans Optimized via Normal integration](https://mlsanity.com/paper/2212.07422/) (2022)
- [TAVA: Template-free Animatable Volumetric Actors](https://mlsanity.com/paper/2206.08929/) (2022)
- [Deep Appearance Models for Face Rendering](https://mlsanity.com/paper/1808.00362/) (2018)
- [DeepCap: Monocular Human Performance Capture Using Weak Supervision](https://mlsanity.com/paper/2003.08325/) (2020)
- [HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion](https://mlsanity.com/paper/2305.06356/) (2023)
- [Neural Human Performer: Learning Generalizable Radiance Fields for Human Performance Rendering](https://mlsanity.com/paper/2109.07448/) (2021)
- [Efficient Neural Radiance Fields for Interactive Free-viewpoint Video](https://mlsanity.com/paper/2112.01517/) (2021)
- [Real-time Deep Dynamic Characters](https://mlsanity.com/paper/2105.01794/) (2021)
- [VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera](https://mlsanity.com/paper/1705.01583/) (2017)
- [Structured Local Radiance Fields for Human Avatar Modeling](https://mlsanity.com/paper/2203.14478/) (2022)
- [HumanNeRF: Efficiently Generated Human Radiance Field from Sparse Inputs](https://mlsanity.com/paper/2112.02789/) (2021)
- [LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration](https://mlsanity.com/paper/2010.12447/) (2020)
- [Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Control](https://mlsanity.com/paper/2106.02019/) (2021)
- [LiveCap: Real-time Human Performance Capture from Monocular Video](https://mlsanity.com/paper/1810.02648/) (2018)

## Built on by (most cited first, 22 of 22)

- [Drivable 3D Gaussian Avatars](https://mlsanity.com/paper/2311.08581/) (2023)
- [DELIFFAS: Deformable Light Fields for Fast Avatar Synthesis](https://mlsanity.com/paper/2310.11449/) (2023)
- [GaussianHeads: End-to-End Learning of Drivable Gaussian Head Avatars from Coarse-to-fine Representations](https://mlsanity.com/paper/2409.11951/) (2024)
- [TriHuman : A Real-time and Controllable Tri-plane Representation for Detailed Human Geometry and Appearance Synthesis](https://mlsanity.com/paper/2312.05161/) (2023)
- [Relightable Full-Body Gaussian Codec Avatars](https://mlsanity.com/paper/2501.14726/) (2025)
- [GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Details](https://mlsanity.com/paper/2405.12420/) (2024)
- [DressRecon: Freeform 4D Human Reconstruction from Monocular Video](https://mlsanity.com/paper/2409.20563/) (2024)
- [Multimodal Latent Diffusion Model for Complex Sewing Pattern Generation](https://mlsanity.com/paper/2412.14453/) (2024)
- [Real-time High-fidelity Gaussian Human Avatars with Position-based Interpolation of Spatially Distributed MLPs](https://mlsanity.com/paper/2504.12909/) (2025)
- [MetaCap: Meta-learning Priors from Multi-View Imagery for Sparse-view Human Performance Capture and Rendering](https://mlsanity.com/paper/2403.18820/) (2024)
- [PICA: Physics-Integrated Clothed Avatar](https://mlsanity.com/paper/2407.05324/) (2024)
- [EgoAvatar: Egocentric View-Driven and Photorealistic Full-body Avatars](https://mlsanity.com/paper/2410.01835/) (2024)
- [Real-time Free-view Human Rendering from Sparse-view RGB Videos using Double Unprojected Textures](https://mlsanity.com/paper/2412.13183/) (2024)
- [EVA: Expressive Virtual Avatars from Multi-view Videos](https://mlsanity.com/paper/2505.15385/) (2025)
- [Interactive Rendering of Relightable and Animatable Gaussian Avatars](https://mlsanity.com/paper/2407.10707/) (2024)
- [AniDress: Animatable Loose-Dressed Avatar from Sparse Views Using Garment Rigging Model](https://mlsanity.com/paper/2401.15348/) (2024)
- [Relightable and Dynamic Gaussian Avatar Reconstruction from Monocular Video](https://mlsanity.com/paper/2512.09335/) (2025)
- [Pb4U-GNet: Resolution-Adaptive Garment Simulation via Propagation-before-Update Graph Network](https://mlsanity.com/paper/2601.15110/) (2026)
- [Garment Animation NeRF with Color Editing](https://mlsanity.com/paper/2407.19774/) (2024)
- [GenFusion: Feed-forward Human Performance Capture via Progressive Canonical Space Updates](https://mlsanity.com/paper/2603.28997/) (2026)
- [Autoregressive Appearance Prediction for 3D Gaussian Avatars](https://mlsanity.com/paper/2604.00928/) (2026)
- [EgoRelight: Egocentric Human Capture and Illumination Recovery for Relightable and Photoreal Avatar Rendering](https://mlsanity.com/paper/2605.28401/) (2026)

## Abstract (arXiv)

Clothing is an important part of human appearance but challenging to model in photorealistic avatars. In this work we present avatars with dynamically moving loose clothing that can be faithfully driven by sparse RGB-D inputs as well as body and face motion. We propose a Neural Iterative Closest Point (N-ICP) algorithm that can efficiently track the coarse garment shape given sparse depth input. Given the coarse tracking results, the input RGB-D images are then remapped to texel-aligned features, which are fed into the drivable avatar models to faithfully reconstruct appearance details. We evaluate our method against recent image-driven synthesis baselines, and conduct a comprehensive analysis of the N-ICP algorithm. We demonstrate that our method can generalize to a novel testing environment, while preserving the ability to produce high-fidelity and faithful clothing dynamics and appearance.