mlsanityData to 7 Oct 2026

Dynamic Surface Function Networks for Clothed Human Bodies

arXiv 2104.03978 ↗First posted 8 Apr 2021cs.CV cs.GRcited by 44

Not on the map. Why?

Abstract (arXiv)

We present a novel method for temporal coherent reconstruction and tracking of clothed humans. Given a monocular RGB-D sequence, we learn a person-specific body model which is based on a dynamic surface function network. To this end, we explicitly model the surface of the person using a multi-layer perceptron (MLP) which is embedded into the canonical space of the SMPL body model. With classical forward rendering, the represented surface can be rasterized using the topology of a template mesh. For each surface point of the template mesh, the MLP is evaluated to predict the actual surface location. To handle pose-dependent deformations, the MLP is conditioned on the SMPL pose parameters. We show that this surface representation as well as the pose parameters can be learned in a self-supervised fashion using the principle of analysis-by-synthesis and differentiable rasterization. As a result, we are able to reconstruct a temporally coherent mesh sequence from the input data. The underlying surface representation can be used to synthesize new animations of the reconstructed person including pose-dependent deformations.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 30 of its 70 references that are in the map. First posted on arXiv 8 Apr 2021.

Builds on

30 of 70 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  3. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  4. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  5. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  6. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  7. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  8. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 573
  9. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  10. 2020Implicit Functions in Feature Space for 3D Shape Reconstruction and Completioncited by 343
  11. 2020ARCH: Animatable Reconstruction of Clothed Humanscited by 257
  12. 2019Learning Shape Templates with Structured Implicit Functionscited by 242
18 more
  1. 2018Video Based Reconstruction of 3D People Modelscited by 237
  2. 2021Neural Articulated Radiance Fieldcited by 196
  3. 2021SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networkscited by 181
  4. 2019Learning to Reconstruct People in Clothing from a Single RGB Cameracited by 168
  5. 2019Tex2Shape: Detailed Full Human Body Geometry From a Single Imagecited by 166
  6. 2020DeepCap: Monocular Human Performance Capture Using Weak Supervisioncited by 162
  7. 2021SMPLicit: Topology-aware Generative Model for Clothed Peoplecited by 154
  8. 2019NASA: Neural Articulated Shape Approximationcited by 149
  9. 2020Combining Implicit Function Learning and Parametric Models for 3D Human Reconstructioncited by 133
  10. 2018Detailed Human Avatars from Monocular Videocited by 109
  11. 2017Detailed, accurate, human shape estimation from clothed 3D scan sequencescited by 89
  12. 2021NPMs: Neural Parametric Models for 3D Deformable Shapescited by 89
  13. 2019Graphonomy: Universal Human Parsing via Graph Transfer Learningcited by 81
  14. 2020PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representationscited by 81
  15. 2016VolumeDeform: Real-time Volumetric Non-rigid Reconstructioncited by 75
  16. 2020Neural Deformation Graphs for Globally-consistent Non-rigid Reconstructioncited by 66
  17. 2020MonoClothCap: Towards Temporally Coherent Clothing Capture from Monocular RGB Videocited by 62
  18. 2013Estimation of Human Body Shape and Posture Under Clothingcited by 2

Built on by

Cited by year

Citing papers per year: 2021: 5; 2022: 15; 2023: 10; 2024: 12; 2025: 2.

44 papers.

Sort:

By area:

  1. 2021Advances in Neural Renderingcited by 365
  2. 2021Neural Head Avatars from Monocular RGB Videoscited by 212
  3. 2022Towards Metrical Reconstruction of Human Facescited by 165
  4. 2022ARAH: Animatable Volume Rendering of Articulated Human SDFscited by 139
  5. 2023Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decompositioncited by 134
  6. 2022Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatarscited by 124
  7. 2022Generalizable Patch-Based Neural Renderingcited by 121
  8. 2021The Power of Points for Modeling Humans in Clothingcited by 90
  9. 2023AvatarReX: Real-time Expressive Full-body Avatarscited by 83
  10. 2021MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Imagescited by 78
  11. 2023HiFi4G: High-Fidelity Human Performance Rendering via Compact Gaussian Splattingcited by 68
  12. 2024Advances in 3D Generation: A Surveycited by 62
32 more
  1. 2022AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatarscited by 55
  2. 2022Learning Implicit Templates for Point-Based Clothed Human Modelingcited by 50
  3. 2023PoseVocab: Learning Joint-structured Pose Embeddings for Human Avatar Modelingcited by 50
  4. 2022PINA: Learning a Personalized Implicit Neural Avatar from a Single RGB-D Video Sequencecited by 46
  5. 2022NeuralHOFusion: Neural Volumetric Rendering under Human-object Interactionscited by 44
  6. 2022CLIP-Actor: Text-Driven Recommendation and Stylization for Animating Human Meshescited by 41
  7. 2022DANBO: Disentangled Articulated Neural Body Representations via Graph Neural Networkscited by 38
  8. 2024MeshAvatar: Learning High-quality Triangular Human Avatars from Multi-view Videoscited by 34
  9. 2021HVTR: Hybrid Volumetric-Textural Rendering for Human Avatarscited by 33
  10. 2022Neural Point-based Shape Modeling of Humans in Challenging Clothingcited by 29
  11. 2023Animatable and Relightable Gaussians for High-fidelity Human Avatar Modelingcited by 27
  12. 2022Neural Shape Deformation Priorscited by 26
  13. 2022AvatarCap: Animatable Avatar Conditioned Monocular Human Volumetric Capturecited by 25
  14. 2022PERGAMO: Personalized 3D Garments from Monocular Videocited by 22
  15. 2024LayGA: Layered Gaussian Avatars for Animatable Clothing Transfercited by 22
  16. 2023CloSET: Modeling Clothed Humans on Continuous Surface with Explicit Template Decompositioncited by 21
  17. 2023CaPhy: Capturing Physical Properties for Animatable Human Avatarscited by 20
  18. 2022Dressing Avatars: Deep Photorealistic Appearance for Physically Simulated Clothingcited by 19
  19. 2024Quark: Real-time, High-resolution, and General Neural View Synthesiscited by 16
  20. 2023SeSDF: Self-evolved Signed Distance Field for Implicit 3D Clothed Human Reconstructioncited by 15
  21. 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 13
  22. 2024ANIM: Accurate Neural Implicit Model for Human Reconstruction from a single RGB-D imagecited by 9
  23. 2024AvatarPose: Avatar-guided 3D Pose Estimation of Close Human Interaction from Sparse Multi-view Videoscited by 7
  24. 2023HeadCraft: Modeling High-Detail Shape Variations for Animated 3DMMscited by 5
  25. 2024Design2Cloth: 3D Cloth Generation from 2D Maskscited by 5
  26. 2024TexVocab: Texture Vocabulary-conditioned Human Avatarscited by 4
  27. 2024FreeCloth: Free-form Generation Enhances Challenging Clothed Human Modelingcited by 2
  28. 2023Invertible Neural Skinningcited by 1
  29. 2024InstantGeoAvatar: Effective Geometry and Appearance Modeling of Animatable Avatars from Monocular Videocited by 0
  30. 2024PocoLoco: A Point Cloud Diffusion Model of Human Shape in Loose Clothingcited by 0
  31. 2024NF3DM: Combining Neural Fields and Deformation Models for 3D Non-Rigid Motion Reconstructioncited by 0
  32. 2025Real-Time Scene Reconstruction using Light Field Probescited by 0