# Surface-Aligned Neural Radiance Fields for Controllable 3D Human Synthesis

arXiv: https://arxiv.org/abs/2201.01683 · HTML: https://mlsanity.com/paper/2201.01683/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 51 references that are in the map.
- First posted on arXiv 5 Jan 2022.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020)
- [Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video](https://mlsanity.com/paper/2012.12247/) (2020)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies](https://mlsanity.com/paper/2105.02872/) (2021)
- [Learning Category-Specific Mesh Reconstruction from Image Collections](https://mlsanity.com/paper/1803.07549/) (2018)
- [Neural Radiance Flow for 4D View Synthesis and Video Processing](https://mlsanity.com/paper/2012.09790/) (2020)
- [ARCH: Animatable Reconstruction of Clothed Humans](https://mlsanity.com/paper/2004.04572/) (2020)
- [Video Based Reconstruction of 3D People Models](https://mlsanity.com/paper/1803.04758/) (2018)
- [Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer](https://mlsanity.com/paper/1908.01210/) (2019)
- [Neural Articulated Radiance Field](https://mlsanity.com/paper/2104.03110/) (2021)
- [Learning to Reconstruct People in Clothing from a Single RGB Camera](https://mlsanity.com/paper/1903.05885/) (2019)
- [Tex2Shape: Detailed Full Human Body Geometry From a Single Image](https://mlsanity.com/paper/1904.08645/) (2019)
- [NASA: Neural Articulated Shape Approximation](https://mlsanity.com/paper/1912.03207/) (2019)
- [Instance-level Human Parsing via Part Grouping Network](https://mlsanity.com/paper/1808.00157/) (2018)
- [Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction](https://mlsanity.com/paper/2007.11432/) (2020)
- [LASR: Learning Articulated Shape Reconstruction from a Monocular Video](https://mlsanity.com/paper/2105.02976/) (2021)
- [SiCloPe: Silhouette-Based Clothed People](https://mlsanity.com/paper/1901.00049/) (2018)
- [Unsupervised Learning of 3D Object Categories from Videos in the Wild](https://mlsanity.com/paper/2103.16552/) (2021)
- [S3: Neural Shape, Skeleton, and Skinning Fields for 3D Human Modeling](https://mlsanity.com/paper/2101.06571/) (2021)
- [Online Adaptation for Consistent Mesh Reconstruction in the Wild](https://mlsanity.com/paper/2012.03196/) (2020)

## Built on by (most cited first, 30 of 41)

- [HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion](https://mlsanity.com/paper/2305.06356/) (2023)
- [Instant Volumetric Head Avatars](https://mlsanity.com/paper/2211.12499/) (2022)
- [ARAH: Animatable Volume Rendering of Articulated Human SDFs](https://mlsanity.com/paper/2210.10036/) (2022)
- [Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decomposition](https://mlsanity.com/paper/2302.11566/) (2023)
- [Generative Neural Articulated Radiance Fields](https://mlsanity.com/paper/2206.14314/) (2022)
- [Deforming Radiance Fields with Cages](https://mlsanity.com/paper/2207.12298/) (2022)
- [Unsupervised Learning of Efficient Geometry-Aware Neural Articulated Representations](https://mlsanity.com/paper/2204.08839/) (2022)
- [3D Gaussian Blendshapes for Head Avatar Animation](https://mlsanity.com/paper/2404.19398/) (2024)
- [Deformable 3D Gaussian Splatting for Animatable Human Avatars](https://mlsanity.com/paper/2312.15059/) (2023)
- [Hand Avatar: Free-Pose Hand Animation and Rendering from Monocular Video](https://mlsanity.com/paper/2211.12782/) (2022)
- [SplatArmor: Articulated Gaussian splatting for animatable humans from monocular RGB videos](https://mlsanity.com/paper/2311.10812/) (2023)
- [Vid2Avatar-Pro: Authentic Avatar from Videos in the Wild via Universal Prior](https://mlsanity.com/paper/2503.01610/) (2025)
- [Neural Novel Actor: Learning a Generalized Animatable Neural Representation for Human Actors](https://mlsanity.com/paper/2208.11905/) (2022)
- [Template-free Articulated Neural Point Clouds for Reposable View Synthesis](https://mlsanity.com/paper/2305.19065/) (2023)
- [A Survey on 3D Human Avatar Modeling -- From Reconstruction to Generation](https://mlsanity.com/paper/2406.04253/) (2024)
- [Template-free Articulated Gaussian Splatting for Real-time Reposable Dynamic View Synthesis](https://mlsanity.com/paper/2412.05570/) (2024)
- [UniHuman: A Unified Model for Editing Human Images in the Wild](https://mlsanity.com/paper/2312.14985/) (2023)
- [RoGSplat: Learning Robust Generalizable Human Gaussian Splatting from Sparse Multi-View Images](https://mlsanity.com/paper/2503.14198/) (2025)
- [Drone-NeRF: Efficient NeRF Based 3D Scene Reconstruction for Large-Scale Drone Survey](https://mlsanity.com/paper/2308.15733/) (2023)
- [SelfNeRF: Fast Training NeRF for Human from Monocular Self-rotating Video](https://mlsanity.com/paper/2210.01651/) (2022)
- [HumanRAM: Feed-forward Human Reconstruction and Animation Model using Transformers](https://mlsanity.com/paper/2506.03118/) (2025)
- [HDhuman: High-quality Human Novel-view Rendering from Sparse Views](https://mlsanity.com/paper/2201.08158/) (2022)
- [VeRi3D: Generative Vertex-based Radiance Fields for 3D Controllable Human Image Synthesis](https://mlsanity.com/paper/2309.04800/) (2023)
- [IntrinsicNGP: Intrinsic Coordinate based Hash Encoding for Human NeRF](https://mlsanity.com/paper/2302.14683/) (2023)
- [Collecting The Puzzle Pieces: Disentangled Self-Driven Human Pose Transfer by Permuting Textures](https://mlsanity.com/paper/2210.01887/) (2022)
- [Explicifying Neural Implicit Fields for Efficient Dynamic Human Avatar Modeling via a Neural Explicit Surface](https://mlsanity.com/paper/2308.05112/) (2023)
- [Neural Rendering of Humans in Novel View and Pose from Monocular Video](https://mlsanity.com/paper/2204.01218/) (2022)
- [Animatable Virtual Humans: Learning pose-dependent human representations in UV space for interactive performance synthesis](https://mlsanity.com/paper/2310.03615/) (2023)
- [NECA: Neural Customizable Human Avatar](https://mlsanity.com/paper/2403.10335/) (2024)
- [Gaussian Control with Hierarchical Semantic Graphs in 3D Human Recovery](https://mlsanity.com/paper/2405.12477/) (2024)

## Abstract (arXiv)

We propose a new method for reconstructing controllable implicit 3D human models from sparse multi-view RGB videos. Our method defines the neural scene representation on the mesh surface points and signed distances from the surface of a human body mesh. We identify an indistinguishability issue that arises when a point in 3D space is mapped to its nearest surface point on a mesh for learning surface-aligned neural scene representation. To address this issue, we propose projecting a point onto a mesh surface using a barycentric interpolation with modified vertex normals. Experiments with the ZJU-MoCap and Human3.6M datasets show that our approach achieves a higher quality in a novel-view and novel-pose synthesis than existing methods. We also demonstrate that our method easily supports the control of body shape and clothes. Project page: https://pfnet-research.github.io/surface-aligned-nerf/.