# MPS-NeRF: Generalizable 3D Human Rendering from Multiview Images

arXiv: https://arxiv.org/abs/2203.16875 · HTML: https://mlsanity.com/paper/2203.16875/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 62 references that are in the map.
- First posted on arXiv 31 Mar 2022.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction](https://mlsanity.com/paper/2104.10078/) (2021)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020)
- [HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video](https://mlsanity.com/paper/2201.04127/) (2022)
- [Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video](https://mlsanity.com/paper/2012.12247/) (2020)
- [Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies](https://mlsanity.com/paper/2105.02872/) (2021)
- [Convolutional Mesh Regression for Single-Image Human Shape Reconstruction](https://mlsanity.com/paper/1905.03244/) (2019)
- [ARCH: Animatable Reconstruction of Clothed Humans](https://mlsanity.com/paper/2004.04572/) (2020)
- [Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies](https://mlsanity.com/paper/1801.01615/) (2018)
- [A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose](https://mlsanity.com/paper/2102.06199/) (2021)
- [Unite the People: Closing the Loop Between 3D and 2D Human Representations](https://mlsanity.com/paper/1701.02468/) (2017)
- [Neural Articulated Radiance Field](https://mlsanity.com/paper/2104.03110/) (2021)
- [GRF: Learning a General Radiance Field for 3D Representation and Rendering](https://mlsanity.com/paper/2010.04595/) (2020)

## Built on by (most cited first, 30 of 53)

- [4D Gaussian Splatting for Real-Time Dynamic Scene Rendering](https://mlsanity.com/paper/2310.08528/) (2023)
- [GauHuman: Articulated Gaussian Splatting from Monocular Human Videos](https://mlsanity.com/paper/2312.02973/) (2023)
- [DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models](https://mlsanity.com/paper/2304.00916/) (2023)
- [SHERF: Generalizable Human NeRF from a Single Image](https://mlsanity.com/paper/2303.12791/) (2023)
- [3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities](https://mlsanity.com/paper/2407.17418/) (2024)
- [IDOL: Instant Photorealistic 3D Human Creation from a Single Image](https://mlsanity.com/paper/2412.14963/) (2024)
- [GoMAvatar: Efficient Animatable Human Modeling from Monocular Video Using Gaussians-on-Mesh](https://mlsanity.com/paper/2404.07991/) (2024)
- [Single-Image 3D Human Digitization with Shape-Guided Diffusion](https://mlsanity.com/paper/2311.09221/) (2023)
- [Generalizable Human Gaussians for Sparse View Synthesis](https://mlsanity.com/paper/2407.12777/) (2024)
- [One-shot Implicit Animatable Avatars with Model-based Priors](https://mlsanity.com/paper/2212.02469/) (2022)
- [HOSNeRF: Dynamic Human-Object-Scene Neural Radiance Fields from a Single Video](https://mlsanity.com/paper/2304.12281/) (2023)
- [Generalizable Human Gaussians from Single-View Image](https://mlsanity.com/paper/2406.06050/) (2024)
- [Neural Novel Actor: Learning a Generalized Animatable Neural Representation for Human Actors](https://mlsanity.com/paper/2208.11905/) (2022)
- [Pseudo-Generalized Dynamic View Synthesis from a Video](https://mlsanity.com/paper/2310.08587/) (2023)
- [Template-Free Single-View 3D Human Digitalization with Diffusion-Guided LRM](https://mlsanity.com/paper/2401.12175/) (2024)
- [Drivable Avatar Clothing: Faithful Full-Body Telepresence with Dynamic Clothing Driven by Sparse RGB-D Input](https://mlsanity.com/paper/2310.05917/) (2023)
- [Mani-GS: Gaussian Splatting Manipulation with Triangular Mesh](https://mlsanity.com/paper/2405.17811/) (2024)
- [ActorsNeRF: Animatable Few-shot Human Rendering with Generalizable NeRFs](https://mlsanity.com/paper/2304.14401/) (2023)
- [Ponymation: Learning Articulated 3D Animal Motions from Unlabeled Online Videos](https://mlsanity.com/paper/2312.13604/) (2023)
- [TransHuman: A Transformer-based Human Representation for Generalizable Neural Human Rendering](https://mlsanity.com/paper/2307.12291/) (2023)
- [AniPortraitGAN: Animatable 3D Portrait Generation from 2D Image Collections](https://mlsanity.com/paper/2309.02186/) (2023)
- [A Survey on 3D Human Avatar Modeling -- From Reconstruction to Generation](https://mlsanity.com/paper/2406.04253/) (2024)
- [LongSplat: Online Generalizable 3D Gaussian Splatting from Long Sequence Images](https://mlsanity.com/paper/2507.16144/) (2025)
- [HumanNeRF-SE: A Simple yet Effective Approach to Animate HumanNeRF with Diverse Poses](https://mlsanity.com/paper/2312.02232/) (2023)
- [RoGSplat: Learning Robust Generalizable Human Gaussian Splatting from Sparse Multi-View Images](https://mlsanity.com/paper/2503.14198/) (2025)
- [Deep learning for 3D human pose estimation and mesh recovery: A survey](https://mlsanity.com/paper/2402.18844/) (2024)
- [GST: Precise 3D Human Body from a Single Image with Gaussian Splatting Transformers](https://mlsanity.com/paper/2409.04196/) (2024)
- [Neural radiance fields in the industrial and robotics domain: applications, research opportunities and use cases](https://mlsanity.com/paper/2308.07118/) (2023)
- [The NeRF Signature: Codebook-Aided Watermarking for Neural Radiance Fields](https://mlsanity.com/paper/2502.19125/) (2025)
- [HumanRAM: Feed-forward Human Reconstruction and Animation Model using Transformers](https://mlsanity.com/paper/2506.03118/) (2025)

## Abstract (arXiv)

There has been rapid progress recently on 3D human rendering, including novel view synthesis and pose animation, based on the advances of neural radiance fields (NeRF). However, most existing methods focus on person-specific training and their training typically requires multi-view videos. This paper deals with a new challenging task -- rendering novel views and novel poses for a person unseen in training, using only multiview images as input. For this task, we propose a simple yet effective method to train a generalizable NeRF with multiview images as conditional input. The key ingredient is a dedicated representation combining a canonical NeRF and a volume deformation scheme. Using a canonical space enables our method to learn shared properties of human and easily generalize to different people. Volume deformation is used to connect the canonical space with input and target images and query image features for radiance and density prediction. We leverage the parametric 3D human model fitted on the input images to derive the deformation, which works quite well in practice when combined with our canonical NeRF. The experiments on both real and synthetic data with the novel view synthesis and pose animation tasks collectively demonstrate the efficacy of our method.