# PINA: Learning a Personalized Implicit Neural Avatar from a Single RGB-D Video Sequence

arXiv: https://arxiv.org/abs/2203.01754 · HTML: https://mlsanity.com/paper/2203.01754/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 71 references that are in the map.
- First posted on arXiv 3 Mar 2022.

## Builds on (most cited first, 30 of 53)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020)
- [SAL: Sign Agnostic Learning of Shapes from Raw Data](https://mlsanity.com/paper/1911.10414/) (2019)
- [Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies](https://mlsanity.com/paper/2105.02872/) (2021)
- [Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion](https://mlsanity.com/paper/2003.01456/) (2020)
- [ICON: Implicit Clothed humans Obtained from Normals](https://mlsanity.com/paper/2112.09127/) (2021)
- [ARCH: Animatable Reconstruction of Clothed Humans](https://mlsanity.com/paper/2004.04572/) (2020)
- [Multi-Garment Net: Learning to Dress 3D People from Images](https://mlsanity.com/paper/1908.06903/) (2019)
- [Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies](https://mlsanity.com/paper/1801.01615/) (2018)
- [Video Based Reconstruction of 3D People Models](https://mlsanity.com/paper/1803.04758/) (2018)
- [PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstruction](https://mlsanity.com/paper/2007.03858/) (2020)
- [I M Avatar: Implicit Morphable Head Avatars from Videos](https://mlsanity.com/paper/2112.07471/) (2021)
- [SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes](https://mlsanity.com/paper/2104.03953/) (2021)
- [HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields](https://mlsanity.com/paper/2106.13228/) (2021)
- [STAR: Sparse Trained Articulated Human Body Regressor](https://mlsanity.com/paper/2008.08535/) (2020)
- [SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks](https://mlsanity.com/paper/2104.03313/) (2021)
- [Learning to Reconstruct People in Clothing from a Single RGB Camera](https://mlsanity.com/paper/1903.05885/) (2019)
- [SMPLicit: Topology-aware Generative Model for Clothed People](https://mlsanity.com/paper/2103.06871/) (2021)

## Built on by (most cited first, 30 of 46)

- [ECON: Explicit Clothed humans Optimized via Normal integration](https://mlsanity.com/paper/2212.07422/) (2022)
- [NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)](https://mlsanity.com/paper/2210.00379/) (2022)
- [ARAH: Animatable Volume Rendering of Articulated Human SDFs](https://mlsanity.com/paper/2210.10036/) (2022)
- [InstantAvatar: Learning Avatars from Monocular Video in 60 Seconds](https://mlsanity.com/paper/2212.10550/) (2022)
- [Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decomposition](https://mlsanity.com/paper/2302.11566/) (2023)
- [SHERF: Generalizable Human NeRF from a Single Image](https://mlsanity.com/paper/2303.12791/) (2023)
- [SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusion](https://mlsanity.com/paper/2311.15855/) (2023)
- [X-Avatar: Expressive Human Avatars](https://mlsanity.com/paper/2303.04805/) (2023)
- [Fast-SNARF: A Fast Deformer for Articulated Neural Fields](https://mlsanity.com/paper/2211.15601/) (2022)
- [AG3D: Learning to Generate 3D Avatars from 2D Image Collections](https://mlsanity.com/paper/2305.02312/) (2023)
- [PoseVocab: Learning Joint-structured Pose Embeddings for Human Avatar Modeling](https://mlsanity.com/paper/2304.13006/) (2023)
- [Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Models](https://mlsanity.com/paper/2406.08475/) (2024)
- [SplatArmor: Articulated Gaussian splatting for animatable humans from monocular RGB videos](https://mlsanity.com/paper/2311.10812/) (2023)
- [MeshAvatar: Learning High-quality Triangular Human Avatars from Multi-view Videos](https://mlsanity.com/paper/2407.08414/) (2024)
- [Animatable Implicit Neural Representations for Creating Realistic Avatars from Videos](https://mlsanity.com/paper/2203.08133/) (2022)
- [Human101: Training 100+FPS Human Gaussians in 100s from 1 View](https://mlsanity.com/paper/2312.15258/) (2023)
- [Neural Point-based Shape Modeling of Humans in Challenging Clothing](https://mlsanity.com/paper/2209.06814/) (2022)
- [Animatable and Relightable Gaussians for High-fidelity Human Avatar Modeling](https://mlsanity.com/paper/2311.16096/) (2023)
- [AvatarCap: Animatable Avatar Conditioned Monocular Human Volumetric Capture](https://mlsanity.com/paper/2207.02031/) (2022)
- [NSF: Neural Surface Fields for Human Modeling from Monocular Depth](https://mlsanity.com/paper/2308.14847/) (2023)
- [GALA: Generating Animatable Layered Assets from a Single Scan](https://mlsanity.com/paper/2401.12979/) (2024)
- [NCHO: Unsupervised Learning for Neural 3D Composition of Humans and Objects](https://mlsanity.com/paper/2305.14345/) (2023)
- [Gen-3Diffusion: Realistic Image-to-3D Generation via 2D & 3D Diffusion Synergy](https://mlsanity.com/paper/2412.06698/) (2024)
- [En3D: An Enhanced Generative Model for Sculpting 3D Humans from 2D Synthetic Data](https://mlsanity.com/paper/2401.01173/) (2024)
- [HiLo: Detailed and Robust 3D Clothed Human Reconstruction with High-and Low-Frequency Information of Parametric Models](https://mlsanity.com/paper/2404.04876/) (2024)
- [PersonNeRF: Personalized Reconstruction from Photo Collections](https://mlsanity.com/paper/2302.08504/) (2023)
- [CanonicalFusion: Generating Drivable 3D Human Avatars from Multiple Images](https://mlsanity.com/paper/2407.04345/) (2024)
- [ANIM: Accurate Neural Implicit Model for Human Reconstruction from a single RGB-D image](https://mlsanity.com/paper/2403.10357/) (2024)
- [Neural-ABC: Neural Parametric Models for Articulated Body with Clothes](https://mlsanity.com/paper/2404.04673/) (2024)
- [MoGA: 3D Generative Avatar Prior for Monocular Gaussian Avatar Reconstruction](https://mlsanity.com/paper/2507.23597/) (2025)

## Abstract (arXiv)

We present a novel method to learn Personalized Implicit Neural Avatars (PINA) from a short RGB-D sequence. This allows non-expert users to create a detailed and personalized virtual copy of themselves, which can be animated with realistic clothing deformations. PINA does not require complete scans, nor does it require a prior learned from large datasets of clothed humans. Learning a complete avatar in this setting is challenging, since only few depth observations are available, which are noisy and incomplete (i.e. only partial visibility of the body per frame). We propose a method to learn the shape and non-rigid deformations via a pose-conditioned implicit surface and a deformation field, defined in canonical space. This allows us to fuse all partial observations into a single consistent canonical representation. Fusion is formulated as a global optimization problem over the pose, shape and skinning parameters. The method can learn neural avatars from real noisy RGB-D sequences for a diverse set of people and clothing styles and these avatars can be animated given unseen motion sequences.