# Animatable Implicit Neural Representations for Creating Realistic Avatars from Videos

arXiv: https://arxiv.org/abs/2203.08133 · HTML: https://mlsanity.com/paper/2203.08133/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 69 of its 97 references that are in the map.
- First posted on arXiv 15 Mar 2022.

## Builds on (most cited first, 30 of 69)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020)
- [HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video](https://mlsanity.com/paper/2201.04127/) (2022)
- [Baking Neural Radiance Fields for Real-Time View Synthesis](https://mlsanity.com/paper/2103.14645/) (2021)
- [Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies](https://mlsanity.com/paper/2105.02872/) (2021)
- [Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion](https://mlsanity.com/paper/2003.01456/) (2020)
- [Neural Point-Based Graphics](https://mlsanity.com/paper/1906.08240/) (2019)
- [Everybody Dance Now](https://mlsanity.com/paper/1808.07371/) (2018)
- [Convolutional Mesh Regression for Single-Image Human Shape Reconstruction](https://mlsanity.com/paper/1905.03244/) (2019)
- [ARCH: Animatable Reconstruction of Clothed Humans](https://mlsanity.com/paper/2004.04572/) (2020)
- [Deferred Neural Rendering: Image Synthesis using Neural Textures](https://mlsanity.com/paper/1904.12356/) (2019)
- [Pose Guided Person Image Generation](https://mlsanity.com/paper/1705.09368/) (2017)

## Built on by (most cited first, 30 of 33)

- [NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)](https://mlsanity.com/paper/2210.00379/) (2022)
- [Relightable and Animatable Neural Avatar from Sparse-View Video](https://mlsanity.com/paper/2308.07903/) (2023)
- [GVA: Reconstructing Vivid 3D Gaussian Avatars from Monocular Videos](https://mlsanity.com/paper/2402.16607/) (2024)
- [LayGA: Layered Gaussian Avatars for Animatable Clothing Transfer](https://mlsanity.com/paper/2405.07319/) (2024)
- [CanonicalFusion: Generating Drivable 3D Human Avatars from Multiple Images](https://mlsanity.com/paper/2407.04345/) (2024)
- [Neural Radiance Fields for the Real World: A Survey](https://mlsanity.com/paper/2501.13104/) (2025)
- [Within the Dynamic Context: Inertia-aware 3D Human Modeling with Pose Sequence](https://mlsanity.com/paper/2403.19160/) (2024)
- [The NeRF Signature: Codebook-Aided Watermarking for Neural Radiance Fields](https://mlsanity.com/paper/2502.19125/) (2025)
- [HumanRAM: Feed-forward Human Reconstruction and Animation Model using Transformers](https://mlsanity.com/paper/2506.03118/) (2025)
- [PICA: Physics-Integrated Clothed Avatar](https://mlsanity.com/paper/2407.05324/) (2024)
- [Motion-Aware Animatable Gaussian Avatars Deblurring](https://mlsanity.com/paper/2411.16758/) (2024)
- [WonderHuman: Hallucinating Unseen Parts in Dynamic 3D Human Reconstruction](https://mlsanity.com/paper/2502.01045/) (2025)
- [EasyVolcap: Accelerating Neural Volumetric Video Research](https://mlsanity.com/paper/2312.06575/) (2023)
- [Motion-Oriented Compositional Neural Radiance Fields for Monocular Dynamic Human Modeling](https://mlsanity.com/paper/2407.11962/) (2024)
- [FIANCEE: Faster Inference of Adversarial Networks via Conditional Early Exits](https://mlsanity.com/paper/2304.10306/) (2023)
- [AniDress: Animatable Loose-Dressed Avatar from Sparse Views Using Garment Rigging Model](https://mlsanity.com/paper/2401.15348/) (2024)
- [WildAvatar: Learning In-the-wild 3D Avatars from the Web](https://mlsanity.com/paper/2407.02165/) (2024)
- [R3-Avatar: Record and Retrieve Temporal Codebook for Reconstructing Photorealistic Human Avatars](https://mlsanity.com/paper/2503.12751/) (2025)
- [PGAHum: Prior-Guided Geometry and Appearance Learning for High-Fidelity Animatable Human Reconstruction](https://mlsanity.com/paper/2404.13862/) (2024)
- [XHand: Real-time Expressive Hand Avatar](https://mlsanity.com/paper/2407.21002/) (2024)
- [TFS-NeRF: Template-Free NeRF for Semantic 3D Reconstruction of Dynamic Scene](https://mlsanity.com/paper/2409.17459/) (2024)
- [RealityAvatar: Towards Realistic Loose Clothing Modeling in Animatable 3D Gaussian Avatars](https://mlsanity.com/paper/2504.01559/) (2025)
- [Inferring Compositional 4D Scenes without Ever Seeing One](https://mlsanity.com/paper/2512.05272/) (2025)
- [HR Human: Modeling Human Avatars with Triangular Mesh and High-Resolution Textures from Videos](https://mlsanity.com/paper/2405.11270/) (2024)
- [InstantGeoAvatar: Effective Geometry and Appearance Modeling of Animatable Avatars from Monocular Video](https://mlsanity.com/paper/2411.01512/) (2024)
- [Fast and Physically-based Neural Explicit Surface for Relightable Human Avatars](https://mlsanity.com/paper/2503.18408/) (2025)
- [Real-Time Animatable 2DGS-Avatars with Detail Enhancement from Monocular Videos](https://mlsanity.com/paper/2505.00421/) (2025)
- [MonoPartNeRF:Human Reconstruction from Monocular Video via Part-Based Neural Radiance Fields](https://mlsanity.com/paper/2508.08798/) (2025)
- [FMGS-Avatar: Mesh-Guided 2D Gaussian Splatting with Foundation Model Priors for 3D Monocular Avatar Reconstruction](https://mlsanity.com/paper/2509.14739/) (2025)
- [Towards Efficient 3D Gaussian Human Avatar Compression: A Prior-Guided Framework](https://mlsanity.com/paper/2510.10492/) (2025)

## Abstract (arXiv)

This paper addresses the challenge of reconstructing an animatable human model from a multi-view video. Some recent works have proposed to decompose a non-rigidly deforming scene into a canonical neural radiance field and a set of deformation fields that map observation-space points to the canonical space, thereby enabling them to learn the dynamic scene from images. However, they represent the deformation field as translational vector field or SE(3) field, which makes the optimization highly under-constrained. Moreover, these representations cannot be explicitly controlled by input motions. Instead, we introduce a pose-driven deformation field based on the linear blend skinning algorithm, which combines the blend weight field and the 3D human skeleton to produce observation-to-canonical correspondences. Since 3D human skeletons are more observable, they can regularize the learning of the deformation field. Moreover, the pose-driven deformation field can be controlled by input skeletal motions to generate new deformation fields to animate the canonical human model. Experiments show that our approach significantly outperforms recent human modeling methods. The code is available at https://zju3dv.github.io/animatable_nerf/.