# Leveraging Intrinsic Properties for Non-Rigid Garment Alignment

arXiv: https://arxiv.org/abs/2308.09519 · HTML: https://mlsanity.com/paper/2308.09519/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 70 references that are in the map.
- First posted on arXiv 18 Aug 2023.

## Builds on (most cited first, 30 of 35)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Multi-Garment Net: Learning to Dress 3D People from Images](https://mlsanity.com/paper/1908.06903/) (2019)
- [SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes](https://mlsanity.com/paper/2104.03953/) (2021)
- [SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks](https://mlsanity.com/paper/2104.03313/) (2021)
- [The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies](https://mlsanity.com/paper/1906.00425/) (2019)
- [Deep Functional Maps: Structured Prediction for Dense Shape Correspondence](https://mlsanity.com/paper/1704.08686/) (2017)
- [Deep Geometric Functional Maps: Robust Feature Learning for Shape Correspondence](https://mlsanity.com/paper/2003.14286/) (2020)
- [Structured Local Radiance Fields for Human Avatar Modeling](https://mlsanity.com/paper/2203.14478/) (2022)
- [LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration](https://mlsanity.com/paper/2010.12447/) (2020)
- [DoubleFusion: Real-time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensor](https://mlsanity.com/paper/1804.06023/) (2018)
- [The Power of Points for Modeling Humans in Clothing](https://mlsanity.com/paper/2109.01137/) (2021)
- [Detailed, accurate, human shape estimation from clothed 3D scan sequences](https://mlsanity.com/paper/1703.04454/) (2017)
- [SIZER: A Dataset and Model for Parsing 3D Clothing and Learning Size Sensitive 3D Clothing](https://mlsanity.com/paper/2007.11610/) (2020)
- [AvatarReX: Real-time Expressive Full-body Avatars](https://mlsanity.com/paper/2305.04789/) (2023)
- [Capturing and Animation of Body and Clothing from Monocular Video](https://mlsanity.com/paper/2210.01868/) (2022)
- [Unsupervised Deep Learning for Structured Shape Matching](https://mlsanity.com/paper/1812.03794/) (2018)
- [Continuous and Orientation-preserving Correspondences via Functional Maps](https://mlsanity.com/paper/1806.04455/) (2018)
- [Deep Shells: Unsupervised Shape Correspondence with Optimal Transport](https://mlsanity.com/paper/2010.15261/) (2020)
- [Smooth Shells: Multi-Scale Shape Registration with Functional Maps](https://mlsanity.com/paper/1905.12512/) (2019)
- [SCALE: Modeling Clothed Humans with a Surface Codec of Articulated Local Elements](https://mlsanity.com/paper/2104.07660/) (2021)
- [Neural Deformation Graphs for Globally-consistent Non-rigid Reconstruction](https://mlsanity.com/paper/2012.01451/) (2020)
- [Modeling Clothing as a Separate Layer for an Animatable Human Avatar](https://mlsanity.com/paper/2106.14879/) (2021)
- [A Survey of Non-Rigid 3D Registration](https://mlsanity.com/paper/2203.07858/) (2022)
- [DPFM: Deep Partial Functional Maps](https://mlsanity.com/paper/2110.09994/) (2021)
- [Learning Implicit Templates for Point-Based Clothed Human Modeling](https://mlsanity.com/paper/2207.06955/) (2022)
- [PoseVocab: Learning Joint-structured Pose Embeddings for Human Avatar Modeling](https://mlsanity.com/paper/2304.13006/) (2023)
- [Non-rigid Point Cloud Registration with Neural Deformation Pyramid](https://mlsanity.com/paper/2205.12796/) (2022)
- [SimulCap : Single-View Human Performance Capture with Cloth Simulation](https://mlsanity.com/paper/1903.06323/) (2019)
- [DEMEA: Deep Mesh Autoencoders for Non-Rigidly Deforming Objects](https://mlsanity.com/paper/1905.10290/) (2019)

## Built on by (most cited first, 6 of 6)

- [Garment3DGen: 3D Garment Stylization and Texture Generation](https://mlsanity.com/paper/2403.18816/) (2024)
- [Gaussian Garments: Reconstructing Simulation-Ready Clothing with Photorealistic Appearance from Multi-View Video](https://mlsanity.com/paper/2409.08189/) (2024)
- [DV-Matcher: Deformation-based Non-Rigid Point Cloud Matching Guided by Pre-trained Visual Features](https://mlsanity.com/paper/2408.08568/) (2024)
- [Design2Cloth: 3D Cloth Generation from 2D Masks](https://mlsanity.com/paper/2404.02686/) (2024)
- [MeshFeat: Multi-Resolution Features for Neural Fields on Meshes](https://mlsanity.com/paper/2407.13592/) (2024)
- [PhyDeformer: High-Quality Non-Rigid Garment Registration with Physics-Awareness](https://mlsanity.com/paper/2501.10455/) (2025)

## Abstract (arXiv)

We address the problem of aligning real-world 3D data of garments, which benefits many applications such as texture learning, physical parameter estimation, generative modeling of garments, etc. Existing extrinsic methods typically perform non-rigid iterative closest point and struggle to align details due to incorrect closest matches and rigidity constraints. While intrinsic methods based on functional maps can produce high-quality correspondences, they work under isometric assumptions and become unreliable for garment deformations which are highly non-isometric. To achieve wrinkle-level as well as texture-level alignment, we present a novel coarse-to-fine two-stage method that leverages intrinsic manifold properties with two neural deformation fields, in the 3D space and the intrinsic space, respectively. The coarse stage performs a 3D fitting, where we leverage intrinsic manifold properties to define a manifold deformation field. The coarse fitting then induces a functional map that produces an alignment of intrinsic embeddings. We further refine the intrinsic alignment with a second neural deformation field for higher accuracy. We evaluate our method with our captured garment dataset, GarmCap. The method achieves accurate wrinkle-level and texture-level alignment and works for difficult garment types such as long coats. Our project page is https://jsnln.github.io/iccv2023_intrinsic/index.html.