# TAVA: Template-free Animatable Volumetric Actors

arXiv: https://arxiv.org/abs/2206.08929 · HTML: https://mlsanity.com/paper/2206.08929/

## Key facts

- Cited by 172 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 50 references that are in the map.
- First posted on arXiv 17 Jun 2022.

## Builds on (most cited first, 29 of 29)

- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video](https://mlsanity.com/paper/2201.04127/) (2022)
- [Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video](https://mlsanity.com/paper/2012.12247/) (2020)
- [Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies](https://mlsanity.com/paper/2105.02872/) (2021)
- [ARCH: Animatable Reconstruction of Clothed Humans](https://mlsanity.com/paper/2004.04572/) (2020)
- [A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose](https://mlsanity.com/paper/2102.06199/) (2021)
- [SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes](https://mlsanity.com/paper/2104.03953/) (2021)
- [Neural Articulated Radiance Field](https://mlsanity.com/paper/2104.03110/) (2021)
- [STAR: Sparse Trained Articulated Human Body Regressor](https://mlsanity.com/paper/2008.08535/) (2020)
- [SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks](https://mlsanity.com/paper/2104.03313/) (2021)
- [NASA: Neural Articulated Shape Approximation](https://mlsanity.com/paper/1912.03207/) (2019)
- [Learning Formation of Physically-Based Face Attributes](https://mlsanity.com/paper/2004.03458/) (2020)
- [Monocular Real-Time Volumetric Performance Capture](https://mlsanity.com/paper/2007.13988/) (2020)
- [Unsupervised Shape and Pose Disentanglement for 3D Meshes](https://mlsanity.com/paper/2007.11341/) (2020)
- [ANR: Articulated Neural Rendering for Virtual Avatars](https://mlsanity.com/paper/2012.12890/) (2020)
- [TexMesh: Reconstructing Detailed Human Texture and Geometry from RGB-D Video](https://mlsanity.com/paper/2008.00158/) (2020)
- [Disentangled Human Body Embedding Based on Deep Hierarchical Neural Network](https://mlsanity.com/paper/1905.05622/) (2019)

## Built on by (most cited first, 30 of 172)

- [Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis](https://mlsanity.com/paper/2308.09713/) (2023)
- [HexPlane: A Fast Representation for Dynamic Scenes](https://mlsanity.com/paper/2301.09632/) (2023)
- [Shape of Motion: 4D Reconstruction from a Single Video](https://mlsanity.com/paper/2407.13764/) (2024)
- [Monocular Dynamic View Synthesis: A Reality Check](https://mlsanity.com/paper/2210.13445/) (2022)
- [3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting](https://mlsanity.com/paper/2312.09228/) (2023)
- [GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians](https://mlsanity.com/paper/2312.02134/) (2023)
- [PointAvatar: Deformable Point-based Head Avatars from Videos](https://mlsanity.com/paper/2212.08377/) (2022)
- [HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion](https://mlsanity.com/paper/2305.06356/) (2023)
- [3DGStream: On-the-Fly Training of 3D Gaussians for Efficient Streaming of Photo-Realistic Free-Viewpoint Videos](https://mlsanity.com/paper/2403.01444/) (2024)
- [NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)](https://mlsanity.com/paper/2210.00379/) (2022)
- [ARAH: Animatable Volume Rendering of Articulated Human SDFs](https://mlsanity.com/paper/2210.10036/) (2022)
- [InstantAvatar: Learning Avatars from Monocular Video in 60 Seconds](https://mlsanity.com/paper/2212.10550/) (2022)
- [Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decomposition](https://mlsanity.com/paper/2302.11566/) (2023)
- [Consistent4D: Consistent 360deg Dynamic Object Generation from Monocular Video](https://mlsanity.com/paper/2311.02848/) (2023)
- [GART: Gaussian Articulated Template Models](https://mlsanity.com/paper/2311.16099/) (2023)
- [Drivable 3D Gaussian Avatars](https://mlsanity.com/paper/2311.08581/) (2023)
- [Human Gaussian Splatting: Real-time Rendering of Animatable Avatars](https://mlsanity.com/paper/2311.17113/) (2023)
- [NerfAcc: Efficient Sampling Accelerates NeRFs](https://mlsanity.com/paper/2305.04966/) (2023)
- [MonoHuman: Animatable Human Neural Field from Monocular Video](https://mlsanity.com/paper/2304.02001/) (2023)
- [AvatarReX: Real-time Expressive Full-body Avatars](https://mlsanity.com/paper/2305.04789/) (2023)
- [NerfAcc: A General NeRF Acceleration Toolbox](https://mlsanity.com/paper/2210.04847/) (2022)
- [Learning Neural Parametric Head Models](https://mlsanity.com/paper/2212.02761/) (2022)
- [Learning Neural Volumetric Representations of Dynamic Humans in Minutes](https://mlsanity.com/paper/2302.12237/) (2023)
- [HiFi4G: High-Fidelity Human Performance Rendering via Compact Gaussian Splatting](https://mlsanity.com/paper/2312.03461/) (2023)
- [NeuralDome: A Neural Modeling Pipeline on Multi-View Human-Object Interactions](https://mlsanity.com/paper/2212.07626/) (2022)
- [X-Avatar: Expressive Human Avatars](https://mlsanity.com/paper/2303.04805/) (2023)
- [Neural Parametric Gaussians for Monocular Non-Rigid Object Reconstruction](https://mlsanity.com/paper/2312.01196/) (2023)
- [Fast-SNARF: A Fast Deformer for Articulated Neural Fields](https://mlsanity.com/paper/2211.15601/) (2022)
- [NeRF-Det: Learning Geometry-Aware Volumetric Representation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2307.14620/) (2023)
- [PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual Observations](https://mlsanity.com/paper/2404.04421/) (2024)

## Abstract (arXiv)

Coordinate-based volumetric representations have the potential to generate photo-realistic virtual avatars from images. However, virtual avatars also need to be controllable even to a novel pose that may not have been observed. Traditional techniques, such as LBS, provide such a function; yet it usually requires a hand-designed body template, 3D scan data, and limited appearance models. On the other hand, neural representation has been shown to be powerful in representing visual details, but are under explored on deforming dynamic articulated actors. In this paper, we propose TAVA, a method to create T emplate-free Animatable Volumetric Actors, based on neural representations. We rely solely on multi-view data and a tracked skeleton to create a volumetric model of an actor, which can be animated at the test time given novel pose. Since TAVA does not require a body template, it is applicable to humans as well as other creatures such as animals. Furthermore, TAVA is designed such that it can recover accurate dense correspondences, making it amenable to content-creation and editing tasks. Through extensive experiments, we demonstrate that the proposed method generalizes well to novel poses as well as unseen views and showcase basic editing capabilities.