# Generalizable Neural Performer: Learning Robust Radiance Fields for Human Novel View Synthesis

arXiv: https://arxiv.org/abs/2204.11798 · HTML: https://mlsanity.com/paper/2204.11798/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 54 references that are in the map.
- First posted on arXiv 25 Apr 2022.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [ARCH: Animatable Reconstruction of Clothed Humans](https://mlsanity.com/paper/2004.04572/) (2020)
- [Deferred Neural Rendering: Image Synthesis using Neural Textures](https://mlsanity.com/paper/1904.12356/) (2019)
- [PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstruction](https://mlsanity.com/paper/2007.03858/) (2020)
- [Stereo Magnification: Learning View Synthesis using Multiplane Images](https://mlsanity.com/paper/1805.09817/) (2018)
- [Panoptic Studio: A Massively Multiview System for Social Interaction Capture](https://mlsanity.com/paper/1612.03153/) (2016)
- [Neural Human Performer: Learning Generalizable Radiance Fields for Human Performance Rendering](https://mlsanity.com/paper/2109.07448/) (2021)
- [H-NeRF: Neural Radiance Fields for Rendering and Temporal Reconstruction of Humans in Motion](https://mlsanity.com/paper/2110.13746/) (2021)
- [Neural Lumigraph Rendering](https://mlsanity.com/paper/2103.11571/) (2021)
- [Deep Implicit Templates for 3D Shape Representation](https://mlsanity.com/paper/2011.14565/) (2020)
- [ANR: Articulated Neural Rendering for Virtual Avatars](https://mlsanity.com/paper/2012.12890/) (2020)
- [Geometry-Guided Progressive NeRF for Generalizable and Efficient Neural Human Rendering](https://mlsanity.com/paper/2112.04312/) (2021)

## Built on by (most cited first, 30 of 40)

- [CelebV-HQ: A Large-Scale Video Facial Attributes Dataset](https://mlsanity.com/paper/2207.12393/) (2022)
- [DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Rendering](https://mlsanity.com/paper/2307.10173/) (2023)
- [MonoHuman: Animatable Human Neural Field from Monocular Video](https://mlsanity.com/paper/2304.02001/) (2023)
- [OmniSVG: A Unified Scalable Vector Graphics Generation Model](https://mlsanity.com/paper/2504.06263/) (2025)
- [Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2407.07532/) (2024)
- [IDOL: Instant Photorealistic 3D Human Creation from a Single Image](https://mlsanity.com/paper/2412.14963/) (2024)
- [MVHumanNet: A Large-scale Dataset of Multi-view Daily Dressing Human Captures](https://mlsanity.com/paper/2312.02963/) (2023)
- [FreeTimeGS: Free Gaussian Primitives at Anytime and Anywhere for Dynamic Scene Reconstruction](https://mlsanity.com/paper/2506.05348/) (2025)
- [RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatars](https://mlsanity.com/paper/2305.13353/) (2023)
- [Generalizable Human Gaussians for Sparse View Synthesis](https://mlsanity.com/paper/2407.12777/) (2024)
- [GM-NeRF: Learning Generalizable Model-based Neural Radiance Fields from Multi-view Images](https://mlsanity.com/paper/2303.13777/) (2023)
- [Neural Image-based Avatars: Generalizable Radiance Fields for Human Avatar Modeling](https://mlsanity.com/paper/2304.04897/) (2023)
- [3DHumanGAN: 3D-Aware Human Image Generation with 3D Pose Mapping](https://mlsanity.com/paper/2212.07378/) (2022)
- [SeSDF: Self-evolved Signed Distance Field for Implicit 3D Clothed Human Reconstruction](https://mlsanity.com/paper/2304.00359/) (2023)
- [HumanNeRF-SE: A Simple yet Effective Approach to Animate HumanNeRF with Diverse Poses](https://mlsanity.com/paper/2312.02232/) (2023)
- [UnitedHuman: Harnessing Multi-Source Data for High-Resolution Human Generation](https://mlsanity.com/paper/2309.14335/) (2023)
- [MVHumanNet++: A Large-scale Dataset of Multi-view Daily Dressing Human Captures with Richer Annotations for 3D Human Digitization](https://mlsanity.com/paper/2505.01838/) (2025)
- [Generalizable Neural Voxels for Fast Human Radiance Fields](https://mlsanity.com/paper/2303.15387/) (2023)
- [SMPLX-Lite: A Realistic and Drivable Avatar Benchmark with Rich Geometry and Texture Annotations](https://mlsanity.com/paper/2405.19609/) (2024)
- [Forge4D: Feed-Forward 4D Human Reconstruction and Interpolation from Uncalibrated Sparse-view Videos](https://mlsanity.com/paper/2509.24209/) (2025)
- [Dynamic Multi-View Scene Reconstruction Using Neural Implicit Surface](https://mlsanity.com/paper/2303.00050/) (2023)
- [KBody: Towards general, robust, and aligned monocular whole-body estimation](https://mlsanity.com/paper/2304.11542/) (2023)
- [Cross-view and Cross-pose Completion for 3D Human Understanding](https://mlsanity.com/paper/2311.09104/) (2023)
- [Advances in Radiance Field for Dynamic Scene: From Neural Field to Gaussian Field](https://mlsanity.com/paper/2505.10049/) (2025)
- [ViStoryBench: Comprehensive Benchmark Suite for Story Visualization](https://mlsanity.com/paper/2505.24862/) (2025)
- [EG-HumanNeRF: Efficient Generalizable Human NeRF Utilizing Human Prior for Sparse View](https://mlsanity.com/paper/2410.12242/) (2024)
- [BVI-CR: A Multi-View Human Dataset for Volumetric Video Compression](https://mlsanity.com/paper/2411.11199/) (2024)
- [NToP: NeRF-Powered Large-scale Dataset Generation for 2D and 3D Human Pose Estimation in Top-View Fisheye Images](https://mlsanity.com/paper/2402.18196/) (2024)
- [Snap-Snap: Taking Two Images to Reconstruct 3D Human Gaussians in Milliseconds](https://mlsanity.com/paper/2508.14892/) (2025)
- [AniPixel: Towards Animatable Pixel-Aligned Human Avatar](https://mlsanity.com/paper/2302.03397/) (2023)

## Abstract (arXiv)

This work targets at using a general deep learning framework to synthesize free-viewpoint images of arbitrary human performers, only requiring a sparse number of camera views as inputs and skirting per-case fine-tuning. The large variation of geometry and appearance, caused by articulated body poses, shapes and clothing types, are the key bottlenecks of this task. To overcome these challenges, we present a simple yet powerful framework, named Generalizable Neural Performer (GNR), that learns a generalizable and robust neural body representation over various geometry and appearance. Specifically, we compress the light fields for novel view human rendering as conditional implicit neural radiance fields from both geometry and appearance aspects. We first introduce an Implicit Geometric Body Embedding strategy to enhance the robustness based on both parametric 3D human body model and multi-view images hints. We further propose a Screen-Space Occlusion-Aware Appearance Blending technique to preserve the high-quality appearance, through interpolating source view appearance to the radiance fields with a relax but approximate geometric guidance. To evaluate our method, we present our ongoing effort of constructing a dataset with remarkable complexity and diversity. The dataset GeneBody-1.0, includes over 360M frames of 370 subjects under multi-view cameras capturing, performing a large variety of pose actions, along with diverse body shapes, clothing, accessories and hairdos. Experiments on GeneBody-1.0 and ZJU-Mocap show better robustness of our methods than recent state-of-the-art generalizable methods among all cross-dataset, unseen subjects and unseen poses settings. We also demonstrate the competitiveness of our model compared with cutting-edge case-specific ones. Dataset, code and model will be made publicly available.