# MonoClothCap: Towards Temporally Coherent Clothing Capture from Monocular RGB Video

arXiv: https://arxiv.org/abs/2009.10711 · HTML: https://mlsanity.com/paper/2009.10711/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Sep 2020.

## Built on by (most cited first, 30 of 62)

- [ICON: Implicit Clothed humans Obtained from Normals](https://mlsanity.com/paper/2112.09127/) (2021)
- [ECON: Explicit Clothed humans Optimized via Normal integration](https://mlsanity.com/paper/2212.07422/) (2022)
- [SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Video](https://mlsanity.com/paper/2201.12792/) (2022)
- [SPEC: Seeing People in the Wild with an Estimated Camera](https://mlsanity.com/paper/2110.00620/) (2021)
- [Structured Local Radiance Fields for Human Avatar Modeling](https://mlsanity.com/paper/2203.14478/) (2022)
- [Animatable Neural Radiance Fields from Monocular RGB Videos](https://mlsanity.com/paper/2106.13629/) (2021)
- [TeCH: Text-guided Reconstruction of Lifelike Clothed Humans](https://mlsanity.com/paper/2308.08545/) (2023)
- [3D Human Pose Estimation via Intuitive Physics](https://mlsanity.com/paper/2303.18246/) (2023)
- [On Self-Contact and Human Pose](https://mlsanity.com/paper/2104.03176/) (2021)
- [SIFU: Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstruction](https://mlsanity.com/paper/2312.06704/) (2023)
- [SCALE: Modeling Clothed Humans with a Surface Codec of Articulated Local Elements](https://mlsanity.com/paper/2104.07660/) (2021)
- [HiFi4G: High-Fidelity Human Performance Rendering via Compact Gaussian Splatting](https://mlsanity.com/paper/2312.03461/) (2023)
- [Modeling Clothing as a Separate Layer for an Animatable Human Avatar](https://mlsanity.com/paper/2106.14879/) (2021)
- [S3: Neural Shape, Skeleton, and Skinning Fields for 3D Human Modeling](https://mlsanity.com/paper/2101.06571/) (2021)
- [Reconstructing Animatable Categories from Videos](https://mlsanity.com/paper/2305.06351/) (2023)
- [Dynamic Surface Function Networks for Clothed Human Bodies](https://mlsanity.com/paper/2104.03978/) (2021)
- [Neural Strands: Learning Hair Geometry and Appearance from Multi-View Images](https://mlsanity.com/paper/2207.14067/) (2022)
- [Learning to Regress Bodies from Images using Differentiable Semantic Rendering](https://mlsanity.com/paper/2110.03480/) (2021)
- [Registering Explicit to Implicit: Towards High-Fidelity Garment mesh Reconstruction from Single Images](https://mlsanity.com/paper/2203.15007/) (2022)
- [High-Fidelity Clothed Avatar Reconstruction from a Single Image](https://mlsanity.com/paper/2304.03903/) (2023)
- [Point-Based Modeling of Human Clothing](https://mlsanity.com/paper/2104.08230/) (2021)
- [AniGS: Animatable Gaussian Avatar from a Single Image with Inconsistent Gaussian Reconstruction](https://mlsanity.com/paper/2412.02684/) (2024)
- [SplatArmor: Articulated Gaussian splatting for animatable humans from monocular RGB videos](https://mlsanity.com/paper/2311.10812/) (2023)
- [Generating Datasets of 3D Garments with Sewing Patterns](https://mlsanity.com/paper/2109.05633/) (2021)
- [State of the Art in Dense Monocular Non-Rigid 3D Reconstruction](https://mlsanity.com/paper/2210.15664/) (2022)
- [Robust Dual Gaussian Splatting for Immersive Human-centric Volumetric Videos](https://mlsanity.com/paper/2409.08353/) (2024)
- [HVH: Learning a Hybrid Neural Volumetric Representation for Dynamic Hair Performance Capture](https://mlsanity.com/paper/2112.06904/) (2021)
- [REC-MV: REconstructing 3D Dynamic Cloth from Monocular Videos](https://mlsanity.com/paper/2305.14236/) (2023)
- [TriHuman : A Real-time and Controllable Tri-plane Representation for Detailed Human Geometry and Appearance Synthesis](https://mlsanity.com/paper/2312.05161/) (2023)
- [Human Performance Capture from Monocular Video in the Wild](https://mlsanity.com/paper/2111.14672/) (2021)

## Abstract (arXiv)

We present a method to capture temporally coherent dynamic clothing deformation from a monocular RGB video input. In contrast to the existing literature, our method does not require a pre-scanned personalized mesh template, and thus can be applied to in-the-wild videos. To constrain the output to a valid deformation space, we build statistical deformation models for three types of clothing: T-shirt, short pants and long pants. A differentiable renderer is utilized to align our captured shapes to the input frames by minimizing the difference in both silhouette, segmentation, and texture. We develop a UV texture growing method which expands the visible texture region of the clothing sequentially in order to minimize drift in deformation tracking. We also extract fine-grained wrinkle detail from the input videos by fitting the clothed surface to the normal maps estimated by a convolutional neural network. Our method produces temporally coherent reconstruction of body and clothing from monocular video. We demonstrate successful clothing capture results from a variety of challenging videos. Extensive quantitative experiments demonstrate the effectiveness of our method on metrics including body pose error and surface reconstruction error of the clothing.