mlsanityData to 7 Oct 2026

PINA: Learning a Personalized Implicit Neural Avatar from a Single RGB-D Video Sequence

arXiv 2203.01754 ↗First posted 3 Mar 2022cs.CVcited by 46

Not on the map. Why?

Abstract (arXiv)

We present a novel method to learn Personalized Implicit Neural Avatars (PINA) from a short RGB-D sequence. This allows non-expert users to create a detailed and personalized virtual copy of themselves, which can be animated with realistic clothing deformations. PINA does not require complete scans, nor does it require a prior learned from large datasets of clothed humans. Learning a complete avatar in this setting is challenging, since only few depth observations are available, which are noisy and incomplete (i.e. only partial visibility of the body per frame). We propose a method to learn the shape and non-rigid deformations via a pose-conditioned implicit surface and a deformation field, defined in canonical space. This allows us to fuse all partial observations into a single consistent canonical representation. Fusion is formulated as a global optimization problem over the pose, shape and skinning parameters. The method can learn neural avatars from real noisy RGB-D sequences for a diverse set of people and clothing styles and these avatars can be animated given unseen motion sequences.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Builds on 53 of its 71 references that are in the map. First posted on arXiv 3 Mar 2022.

Builds on

53 of 71 references.

References · 53

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Improved Techniques for Training GANscited by 2,938
  3. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  4. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  5. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  6. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  7. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  8. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  9. 2020Implicit Geometric Regularization for Learning Shapescited by 707
  10. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
  11. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  12. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
38 more
  1. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 573
  2. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  3. 2019SAL: Sign Agnostic Learning of Shapes from Raw Datacited by 389
  4. 2021Animatable Neural Radiance Fields for Modeling Dynamic Human Bodiescited by 361
  5. 2020Implicit Functions in Feature Space for 3D Shape Reconstruction and Completioncited by 343
  6. 2021ICON: Implicit Clothed humans Obtained from Normalscited by 283
  7. 2020ARCH: Animatable Reconstruction of Clothed Humanscited by 257
  8. 2019Multi-Garment Net: Learning to Dress 3D People from Imagescited by 244
  9. 2018Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodiescited by 239
  10. 2018Video Based Reconstruction of 3D People Modelscited by 237
  11. 2020PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstructioncited by 231
  12. 2021I M Avatar: Implicit Morphable Head Avatars from Videoscited by 230
  13. 2021SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapescited by 221
  14. 2021HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fieldscited by 210
  15. 2020STAR: Sparse Trained Articulated Human Body Regressorcited by 194
  16. 2021SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networkscited by 181
  17. 2019Learning to Reconstruct People in Clothing from a Single RGB Cameracited by 168
  18. 2021SMPLicit: Topology-aware Generative Model for Clothed Peoplecited by 154
  19. 2019NASA: Neural Articulated Shape Approximationcited by 149
  20. 2020TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Stylecited by 148
  21. 2020Combining Implicit Function Learning and Parametric Models for 3D Human Reconstructioncited by 133
  22. 2021Real-time Deep Dynamic Characterscited by 133
  23. 2020Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstructioncited by 112
  24. 2021Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Controlcited by 106
  25. 2021Neural-GIF: Neural Generalized Implicit Functions for Animating People in Clothingcited by 102
  26. 2018DoubleFusion: Real-time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensorcited by 95
  27. 2017Detailed, accurate, human shape estimation from clothed 3D scan sequencescited by 89
  28. 2020Human Body Model Fitting by Learned Gradient Descentcited by 81
  29. 2021Self-Supervised Collision Handling via Generative 3D Garment Models for Virtual Try-Oncited by 81
  30. 2021MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Imagescited by 78
  31. 2020Monocular Real-Time Volumetric Performance Capturecited by 74
  32. 2022gDNA: Towards Generative Detailed Neural Avatarscited by 72
  33. 2018GarNet: A Two-Stream Network for Fast and Accurate 3D Cloth Drapingcited by 69
  34. 2020Neural Deformation Graphs for Globally-consistent Non-rigid Reconstructioncited by 66
  35. 2020ANR: Articulated Neural Rendering for Virtual Avatarscited by 55
  36. 2021Reconstructing 3D Human Pose by Watching Humans in the Mirrorcited by 47
  37. 2021Dynamic Surface Function Networks for Clothed Human Bodiescited by 44
  38. 2020DeePSD: Automatic Deep Skinning And Pose Space Deformation For 3D Garment Animationcited by 43

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 14; 2024: 16; 2025: 7; 2026: 1.

46 papers.

Sort:

By area:

  1. 2022ECON: Explicit Clothed humans Optimized via Normal integrationcited by 199
  2. 2022NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)cited by 156
  3. 2022ARAH: Animatable Volume Rendering of Articulated Human SDFscited by 139
  4. 2022InstantAvatar: Learning Avatars from Monocular Video in 60 Secondscited by 138
  5. 2023Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decompositioncited by 134
  6. 2023SHERF: Generalizable Human NeRF from a Single Imagecited by 88
  7. 2023SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusioncited by 72
  8. 2023X-Avatar: Expressive Human Avatarscited by 66
  9. 2022Fast-SNARF: A Fast Deformer for Articulated Neural Fieldscited by 55
  10. 2023AG3D: Learning to Generate 3D Avatars from 2D Image Collectionscited by 55
  11. 2023PoseVocab: Learning Joint-structured Pose Embeddings for Human Avatar Modelingcited by 50
  12. 2024Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Modelscited by 44
34 more
  1. 2023SplatArmor: Articulated Gaussian splatting for animatable humans from monocular RGB videoscited by 35
  2. 2024MeshAvatar: Learning High-quality Triangular Human Avatars from Multi-view Videoscited by 34
  3. 2022Animatable Implicit Neural Representations for Creating Realistic Avatars from Videoscited by 33
  4. 2023Human101: Training 100+FPS Human Gaussians in 100s from 1 Viewcited by 32
  5. 2022Neural Point-based Shape Modeling of Humans in Challenging Clothingcited by 29
  6. 2023Animatable and Relightable Gaussians for High-fidelity Human Avatar Modelingcited by 27
  7. 2022AvatarCap: Animatable Avatar Conditioned Monocular Human Volumetric Capturecited by 25
  8. 2023NSF: Neural Surface Fields for Human Modeling from Monocular Depthcited by 21
  9. 2024GALA: Generating Animatable Layered Assets from a Single Scancited by 21
  10. 2023NCHO: Unsupervised Learning for Neural 3D Composition of Humans and Objectscited by 17
  11. 2024Gen-3Diffusion: Realistic Image-to-3D Generation via 2D & 3D Diffusion Synergycited by 16
  12. 2024En3D: An Enhanced Generative Model for Sculpting 3D Humans from 2D Synthetic Datacited by 14
  13. 2024HiLo: Detailed and Robust 3D Clothed Human Reconstruction with High-and Low-Frequency Information of Parametric Modelscited by 13
  14. 2023PersonNeRF: Personalized Reconstruction from Photo Collectionscited by 12
  15. 2024CanonicalFusion: Generating Drivable 3D Human Avatars from Multiple Imagescited by 12
  16. 2024ANIM: Accurate Neural Implicit Model for Human Reconstruction from a single RGB-D imagecited by 9
  17. 2024Neural-ABC: Neural Parametric Models for Articulated Body with Clothescited by 9
  18. 2025MoGA: 3D Generative Avatar Prior for Monocular Gaussian Avatar Reconstructioncited by 8
  19. 2024SMPLX-Lite: A Realistic and Drivable Avatar Benchmark with Rich Geometry and Texture Annotationscited by 7
  20. 2024AvatarPose: Avatar-guided 3D Pose Estimation of Close Human Interaction from Sparse Multi-view Videoscited by 7
  21. 2025HumanGif: Single-View Human Diffusion with Generative Priorcited by 7
  22. 2024TexVocab: Texture Vocabulary-conditioned Human Avatarscited by 4
  23. 2025Advances in Radiance Field for Dynamic Scene: From Neural Field to Gaussian Fieldcited by 4
  24. 2024Gaussian Control with Hierarchical Semantic Graphs in 3D Human Recoverycited by 3
  25. 2024WildAvatar: Learning In-the-wild 3D Avatars from the Webcited by 3
  26. 2023Mesh Strikes Back: Fast and Efficient Human Reconstruction from RGB videoscited by 2
  27. 2025Efficient Neural Video Representation with Temporally Coherent Modulationcited by 2
  28. 2024FAMOUS: High-Fidelity Monocular 3D Human Digitization Using View Synthesiscited by 1
  29. 2023UVA: Towards Unified Volumetric Avatar for View Synthesis, Pose rendering, Geometry and Texture Editingcited by 0
  30. 2024InstantGeoAvatar: Effective Geometry and Appearance Modeling of Animatable Avatars from Monocular Videocited by 0
  31. 2025Link to the Past: Temporal Propagation for Fast 3D Human Reconstruction from Monocular Videocited by 0
  32. 2025ReMu: Reconstructing Multi-layer 3D Clothed Human from Image Layerscited by 0
  33. 2025MExECON: Multi-view Extended Explicit Clothed humans Optimized via Normal integrationcited by 0
  34. 2026DSAR: Dual-Stream Autoregressive Modeling of Temporal Cloth Dynamics for Photorealistic Animatable Avatarscited by 0