mlsanityData to 7 Oct 2026

PocoLoco: A Point Cloud Diffusion Model of Human Shape in Loose Clothing

arXiv 2411.04249 ↗First posted 6 Nov 2024cs.CVcited by 0

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Modeling a human avatar that can plausibly deform to articulations is an active area of research. We present PocoLoco -- the first template-free, point-based, pose-conditioned generative model for 3D humans in loose clothing. We motivate our work by noting that most methods require a parametric model of the human body to ground pose-dependent deformations. Consequently, they are restricted to modeling clothing that is topologically similar to the naked body and do not extend well to loose clothing. The few methods that attempt to model loose clothing typically require either canonicalization or a UV-parameterization and need to address the challenging problem of explicitly estimating correspondences for the deforming clothes. In this work, we formulate avatar clothing deformation as a conditional point-cloud generation task within the denoising diffusion framework. Crucially, our framework operates directly on unordered point clouds, eliminating the need for a parametric model or a clothing template. This also enables a variety of practical applications, such as point-cloud completion and pose-based editing -- important features for virtual human animation. As current datasets for human avatars in loose clothing are far too small for training diffusion models, we release a dataset of two subjects performing various poses in loose clothing with a total of 75K point clouds. By contributing towards tackling the challenging task of effectively modeling loose clothing and expanding the available data for training these models, we aim to set the stage for further innovation in digital humans. The source code is available at https://github.com/sidsunny/pocoloco .

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 35 of its 44 references that are in the map. First posted on arXiv 6 Nov 2024.

Builds on

35 of 44 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  2. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct

Other references · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  7. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  8. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  9. 2014Conditional Generative Adversarial Netscited by 2,152
  10. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  11. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  12. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
21 more
  1. 2019Point-Voxel CNN for Efficient 3D Deep Learningcited by 343
  2. 2022NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstructioncited by 311
  3. 2021ICON: Implicit Clothed humans Obtained from Normalscited by 283
  4. 2021SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapescited by 221
  5. 2023GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussianscited by 190
  6. 2021SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networkscited by 181
  7. 2020DeepCap: Monocular Human Performance Capture Using Weak Supervisioncited by 162
  8. 2020TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Stylecited by 148
  9. 2021Real-time Deep Dynamic Characterscited by 133
  10. 2021HumanNeRF: Efficiently Generated Human Radiance Field from Sparse Inputscited by 112
  11. 2021Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Controlcited by 106
  12. 2021The Power of Points for Modeling Humans in Clothingcited by 90
  13. 2021NPMs: Neural Parametric Models for 3D Deformable Shapescited by 89
  14. 2021MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Imagescited by 78
  15. 2022KeypointNeRF: Generalizing Image-based Volumetric Avatars using Relative Spatial Encoding of Keypointscited by 77
  16. 2017MonoPerfCap: Human Performance Capture from Monocular Videocited by 63
  17. 2021Dynamic Surface Function Networks for Clothed Human Bodiescited by 44
  18. 2023Dynamic Point Fieldscited by 37
  19. 2022Neural Point-based Shape Modeling of Humans in Challenging Clothingcited by 29
  20. 2023GM-NeRF: Learning Generalizable Model-based Neural Radiance Fields from Multi-view Imagescited by 24
  21. 2022BodyMap: Learning Full-Body Dense Correspondence Mapcited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025D-Garment: Physically Grounded Latent Diffusion for Dynamic Garment Deformations
  2. 2026DiT-Garment: Garment Dynamics with Diffusion Transformers
  3. 2025DiffusedWrinkles: A Diffusion-Based Model for Data-Driven Garment Animation
  4. 2023AvatarFusion: Zero-shot Generation of Clothing-Decoupled 3D Avatars Using 2D Diffusion
  5. 2025Generative Human Geometry Distribution
  6. 2022Garment Avatars: Realistic Cloth Driving using Pattern Registration
  7. 2024Multi-hypotheses Conditioned Point Cloud Diffusion for 3D Human Reconstruction from Occluded Images
  8. 2025Single View Garment Reconstruction Using Diffusion Mapping Via Pattern Coordinates