# Vid2Actor: Free-viewpoint Animatable Person Synthesis from Video in the Wild

arXiv: https://arxiv.org/abs/2012.12884 · HTML: https://mlsanity.com/paper/2012.12884/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Dec 2020.

## Built on by (most cited first, 30 of 35)

- [HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video](https://mlsanity.com/paper/2201.04127/) (2022)
- [NeuMan: Neural Human Radiance Field from a Single Video](https://mlsanity.com/paper/2203.12575/) (2022)
- [Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation](https://mlsanity.com/paper/2205.04334/) (2022)
- [SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes](https://mlsanity.com/paper/2104.03953/) (2021)
- [GauHuman: Articulated Gaussian Splatting from Monocular Human Videos](https://mlsanity.com/paper/2312.02973/) (2023)
- [HUGS: Human Gaussian Splats](https://mlsanity.com/paper/2311.17910/) (2023)
- [SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Video](https://mlsanity.com/paper/2201.12792/) (2022)
- [Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Control](https://mlsanity.com/paper/2106.02019/) (2021)
- [ASH: Animatable Gaussian Splats for Efficient and Photoreal Human Rendering](https://mlsanity.com/paper/2312.05941/) (2023)
- [MonoHuman: Animatable Human Neural Field from Monocular Video](https://mlsanity.com/paper/2304.02001/) (2023)
- [SplatArmor: Articulated Gaussian splatting for animatable humans from monocular RGB videos](https://mlsanity.com/paper/2311.10812/) (2023)
- [Common Pets in 3D: Dynamic New-View Synthesis of Real-Life Deformable Categories](https://mlsanity.com/paper/2211.03889/) (2022)
- [TriHuman : A Real-time and Controllable Tri-plane Representation for Detailed Human Geometry and Appearance Synthesis](https://mlsanity.com/paper/2312.05161/) (2023)
- [ActorsNeRF: Animatable Few-shot Human Rendering with Generalizable NeRFs](https://mlsanity.com/paper/2304.14401/) (2023)
- [Audio-driven Neural Gesture Reenactment with Video Motion Graphs](https://mlsanity.com/paper/2207.11524/) (2022)
- [Realistic Full-Body Anonymization with Surface-Guided GANs](https://mlsanity.com/paper/2201.02193/) (2022)
- [FlexNeRF: Photorealistic Free-viewpoint Rendering of Moving Humans from Sparse Views](https://mlsanity.com/paper/2303.14368/) (2023)
- [Rendering Humans from Object-Occluded Monocular Videos](https://mlsanity.com/paper/2308.04622/) (2023)
- [DreaMo: Articulated 3D Reconstruction From A Single Casual Video](https://mlsanity.com/paper/2312.02617/) (2023)
- [SqueezeMe: Mobile-Ready Distillation of Gaussian Full-Body Avatars](https://mlsanity.com/paper/2412.15171/) (2024)
- [Generalizable Neural Voxels for Fast Human Radiance Fields](https://mlsanity.com/paper/2303.15387/) (2023)
- [FreeMan: Towards Benchmarking 3D Human Pose Estimation under Real-World Conditions](https://mlsanity.com/paper/2309.05073/) (2023)
- [Real-time volumetric rendering of dynamic humans](https://mlsanity.com/paper/2303.11898/) (2023)
- [MIGS: Multi-Identity Gaussian Splatting via Tensor Decomposition](https://mlsanity.com/paper/2407.07284/) (2024)
- [Mesh Strikes Back: Fast and Efficient Human Reconstruction from RGB videos](https://mlsanity.com/paper/2303.08808/) (2023)
- [UMFuse: Unified Multi View Fusion for Human Editing applications](https://mlsanity.com/paper/2211.10157/) (2022)
- [Envisioning a Next Generation Extended Reality Conferencing System with Efficient Photorealistic Human Rendering](https://mlsanity.com/paper/2306.16541/) (2023)
- [PixelHuman: Animatable Neural Radiance Fields from Few Images](https://mlsanity.com/paper/2307.09070/) (2023)
- [Semantic-Human: Neural Rendering of Humans from Monocular Video with Human Parsing](https://mlsanity.com/paper/2308.09894/) (2023)
- [TEDRA: Text-based Editing of Dynamic and Photoreal Actors](https://mlsanity.com/paper/2408.15995/) (2024)

## Abstract (arXiv)

Given an "in-the-wild" video of a person, we reconstruct an animatable model of the person in the video. The output model can be rendered in any body pose to any camera view, via the learned controls, without explicit 3D mesh reconstruction. At the core of our method is a volumetric 3D human representation reconstructed with a deep network trained on input video, enabling novel pose/view synthesis. Our method is an advance over GAN-based image-to-image translation since it allows image synthesis for any pose and camera via the internal 3D representation, while at the same time it does not require a pre-rigged model or ground truth meshes for training, as in mesh-based learning. Experiments validate the design choices and yield results on synthetic data and on real videos of diverse people performing unconstrained activities (e.g. dancing or playing tennis). Finally, we demonstrate motion re-targeting and bullet-time rendering with the learned models.