mlsanityData to 14 Oct 2026

H4D: Human 4D Modeling by Learning Neural Compositional Representation

arXiv 2203.01247 ↗First posted 2 Mar 2022cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

Despite the impressive results achieved by deep learning based 3D reconstruction, the techniques of directly learning to model 4D human captures with detailed geometry have been less studied. This work presents a novel framework that can effectively learn a compact and compositional representation for dynamic human by exploiting the human body prior from the widely used SMPL parametric model. Particularly, our representation, named H4D, represents a dynamic 3D human over a temporal span with the SMPL parameters of shape and initial pose, and latent codes encoding motion and auxiliary information. A simple yet effective linear motion model is proposed to provide a rough and regularized motion estimation, followed by per-frame compensation for pose and geometry details with the residual encoded in the auxiliary code. Technically, we introduce novel GRU-based architectures to facilitate learning and improve the representation capability. Extensive experiments demonstrate our method is not only efficacy in recovering dynamic human with accurate motion and detailed geometry, but also amenable to various 4D human related tasks, including motion retargeting, motion completion and future prediction. Please check out the project page for video and code: https://boyanjiang.github.io/H4D/.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 58 of its 74 references that are in the map. First posted on arXiv 2 Mar 2022.

Builds on

58 of 74 references.

References · 58

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  3. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  4. 2018Neural Ordinary Differential Equationscited by 3,779
  5. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  6. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  7. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  8. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  9. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 979
  10. 2017End-to-end Recovery of Human Shape and Posecited by 885
  11. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  12. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 784
38 more
  1. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 695
  2. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 651
  3. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  4. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 627
  5. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 620
  6. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 588
  7. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  8. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  9. 2020Local Implicit Grid Representations for 3D Scenescited by 373
  10. 2020Implicit Functions in Feature Space for 3D Shape Reconstruction and Completioncited by 343
  11. 2017On human motion prediction using recurrent neural networkscited by 338
  12. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 324
  13. 2020Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstructioncited by 321
  14. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  15. 2018Learning 3D Human Dynamics from Videocited by 316
  16. 2019Learning Trajectory Dependencies for Human Motion Predictioncited by 246
  17. 2020PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstructioncited by 231
  18. 2018Learning to Estimate 3D Human Pose and Shape from a Single Color Imagecited by 226
  19. 2017Unite the People: Closing the Loop Between 3D and 2D Human Representationscited by 205
  20. 2017Single-Shot Multi-Person 3D Pose Estimation From Monocular RGBcited by 195
  21. 2021SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networkscited by 181
  22. 2019DIST: Rendering Deep Implicit Signed Distance Function with Differentiable Sphere Tracingcited by 180
  23. 2017HP-GAN: Probabilistic 3D human motion prediction via GANcited by 176
  24. 2016Learning a Predictable and Generative Vector Representation for Objectscited by 173
  25. 2019Learning to Reconstruct People in Clothing from a Single RGB Cameracited by 168
  26. 2019Tex2Shape: Detailed Full Human Body Geometry From a Single Imagecited by 166
  27. 2020Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Videocited by 163
  28. 2021SMPLicit: Topology-aware Generative Model for Clothed Peoplecited by 155
  29. 2020A Spatio-temporal Transformer for 3D Human Motion Predictioncited by 133
  30. 2020Combining Implicit Function Learning and Parametric Models for 3D Human Reconstructioncited by 133
  31. 2020Points2Surf: Learning Implicit Surfaces from Point Cloud Patchescited by 122
  32. 2020BCNet: Learning Body and Cloth Shape from A Single Imagecited by 120
  33. 2018Detailed Human Avatars from Monocular Videocited by 109
  34. 2019Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformationcited by 103
  35. 2019Structured Prediction Helps 3D Human Motion Modellingcited by 96
  36. 2021NPMs: Neural Parametric Models for 3D Deformable Shapescited by 90
  37. 2019360-Degree Textures of People in Clothing from a Single Imagecited by 75
  38. 2018Photo Wake-Up: 3D Character Animation from a Single Photocited by 60

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 4; 2024: 5; 2025: 2; 2026: 2.

15 papers.

Sort:

  1. 2023Generalizing Neural Human Fitting to Unseen Poses With Articulated SE(3) Equivariancecited by 14
  2. 20234DHumanOutfit: a multi-subject 4D dataset of human motion sequences in varying outfits exhibiting large displacementscited by 12
  3. 2023PointHPS: Cascaded 3D Human Pose and Shape Estimation from Point Cloudscited by 9
  4. 2025ETCH: Generalizing Body Fitting to Clothed Humans via Equivariant Tightnesscited by 9
  5. 2022LoRD: Local 4D Implicit Representation for High-Fidelity Dynamic Human Modelingcited by 8
  6. 2024DynoSurf: Neural Deformation-based Temporally Consistent Dynamic Surface Reconstructioncited by 8
  7. 2023Correspondence-free online human motion retargetingcited by 7
  8. 2024Learning Coherent Matrixized Representation in Latent Space for Volumetric 4D Generationcited by 4
  9. 2024A Riemannian Approach for Spatiotemporal Analysis and Generation of 4D Tree-shaped Structurescited by 4
  10. 2022Representing motion as a sequence of latent primitives, a flexible approach for human motion modellingcited by 3
  11. 2024Gaussian Control with Hierarchical Semantic Graphs in 3D Human Recoverycited by 3
  12. 2024NF3DM: Combining Neural Fields and Deformation Models for 3D Non-Rigid Motion Reconstructioncited by 1
3 more
  1. 2025Quality assessment of 3D human animation: Subjective and objective evaluationcited by 1
  2. 2026Learning Neural Deformation Representation for 4D Dynamic Shape Generationcited by 1
  3. 2026TROPHIES: Temporal Reconstruction of Places, Humans, and Cameras from Multi-view Videoscited by 0