# Generalizable Human Gaussians for Sparse View Synthesis

arXiv: https://arxiv.org/abs/2407.12777 · HTML: https://mlsanity.com/paper/2407.12777/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 59 references that are in the map.
- First posted on arXiv 17 Jul 2024.

## Builds on (most cited first, 30 of 48)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation](https://mlsanity.com/paper/2309.16653/) (2023)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies](https://mlsanity.com/paper/2105.02872/) (2021)
- [Neural Point-Based Graphics](https://mlsanity.com/paper/1906.08240/) (2019)
- [Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensors](https://mlsanity.com/paper/2105.01859/) (2021)
- [A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose](https://mlsanity.com/paper/2102.06199/) (2021)
- [ECON: Explicit Clothed humans Optimized via Normal integration](https://mlsanity.com/paper/2212.07422/) (2022)
- [Neural Rays for Occlusion-aware Image-based Rendering](https://mlsanity.com/paper/2107.13421/) (2021)
- [GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians](https://mlsanity.com/paper/2312.02134/) (2023)
- [GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesis](https://mlsanity.com/paper/2312.02155/) (2023)
- [Neural Human Performer: Learning Generalizable Radiance Fields for Human Performance Rendering](https://mlsanity.com/paper/2109.07448/) (2021)
- [HUGS: Human Gaussian Splats](https://mlsanity.com/paper/2311.17910/) (2023)
- [Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction](https://mlsanity.com/paper/2007.11432/) (2020)
- [Drivable 3D Gaussian Avatars](https://mlsanity.com/paper/2311.08581/) (2023)
- [Is Attention All That NeRF Needs?](https://mlsanity.com/paper/2207.13298/) (2022)
- [HumanNeRF: Efficiently Generated Human Radiance Field from Sparse Inputs](https://mlsanity.com/paper/2112.02789/) (2021)
- [LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration](https://mlsanity.com/paper/2010.12447/) (2020)
- [Human Gaussian Splatting: Real-time Rendering of Animatable Avatars](https://mlsanity.com/paper/2311.17113/) (2023)
- [TeCH: Text-guided Reconstruction of Lifelike Clothed Humans](https://mlsanity.com/paper/2308.08545/) (2023)
- [ASH: Animatable Gaussian Splats for Efficient and Photoreal Human Rendering](https://mlsanity.com/paper/2312.05941/) (2023)

## Built on by (most cited first, 30 of 40)

- [SplatAD: Real-Time Lidar and Camera Rendering with 3D Gaussian Splatting for Autonomous Driving](https://mlsanity.com/paper/2411.16816/) (2024)
- [Vid2Avatar-Pro: Authentic Avatar from Videos in the Wild via Universal Prior](https://mlsanity.com/paper/2503.01610/) (2025)
- [GPS-Gaussian+: Generalizable Pixel-wise 3D Gaussian Splatting for Real-Time Human-Scene Rendering from Sparse Views](https://mlsanity.com/paper/2411.11363/) (2024)
- [GAS: Generative Avatar Synthesis from a Single Image](https://mlsanity.com/paper/2502.06957/) (2025)
- [RoGSplat: Learning Robust Generalizable Human Gaussian Splatting from Sparse Multi-View Images](https://mlsanity.com/paper/2503.14198/) (2025)
- [SIGMAN:Scaling 3D Human Gaussian Generation with Millions of Assets](https://mlsanity.com/paper/2504.06982/) (2025)
- [Real-time High-fidelity Gaussian Human Avatars with Position-based Interpolation of Spatially Distributed MLPs](https://mlsanity.com/paper/2504.12909/) (2025)
- [LIFe-GoM: Generalizable Human Rendering with Learned Iterative Feedback Over Multi-Resolution Gaussians-on-Mesh](https://mlsanity.com/paper/2502.09617/) (2025)
- [TaoAvatar: Real-Time Lifelike Full-Body Talking Avatars for Augmented Reality via 3D Gaussian Splatting](https://mlsanity.com/paper/2503.17032/) (2025)
- [MVHumanNet++: A Large-scale Dataset of Multi-view Daily Dressing Human Captures with Richer Annotations for 3D Human Digitization](https://mlsanity.com/paper/2505.01838/) (2025)
- [MPMAvatar: Learning 3D Gaussian Avatars with Accurate and Robust Physics-Based Dynamics](https://mlsanity.com/paper/2510.01619/) (2025)
- [Real-time Free-view Human Rendering from Sparse-view RGB Videos using Double Unprojected Textures](https://mlsanity.com/paper/2412.13183/) (2024)
- [3D-Consistent Human Avatars with Sparse Inputs via Gaussian Splatting and Contrastive Learning](https://mlsanity.com/paper/2408.09663/) (2024)
- [GarmentCrafter: Progressive Novel View Synthesis for Single-View 3D Garment Reconstruction and Editing](https://mlsanity.com/paper/2503.08678/) (2025)
- [SignSplat: Rendering Sign Language via Gaussian Splatting](https://mlsanity.com/paper/2505.02108/) (2025)
- [EVA-Gaussian: 3D Gaussian-based Real-time Human Novel View Synthesis under Diverse Multi-view Camera Settings](https://mlsanity.com/paper/2410.01425/) (2024)
- [Sparse-View 3D Reconstruction: Recent Advances and Open Challenges](https://mlsanity.com/paper/2507.16406/) (2025)
- [MV-Performer: Taming Video Diffusion Model for Faithful and Synchronized Multi-view Performer Synthesis](https://mlsanity.com/paper/2510.07190/) (2025)
- [Parametric Gaussian Human Model: Generalizable Prior for Efficient and Realistic Human Avatar Modeling](https://mlsanity.com/paper/2506.06645/) (2025)
- [Snap-Snap: Taking Two Images to Reconstruct 3D Human Gaussians in Milliseconds](https://mlsanity.com/paper/2508.14892/) (2025)
- [HART: Human Aligned Reconstruction Transformer](https://mlsanity.com/paper/2509.26621/) (2025)
- [FastAnimate: Towards Learnable Template Construction and Pose Deformation for Fast 3D Human Avatar Animation](https://mlsanity.com/paper/2512.01444/) (2025)
- [Real-Time Human Frontal View Synthesis from a Single Image](https://mlsanity.com/paper/2603.15433/) (2026)
- [High-Fidelity Mobile Avatars with Pruned Local Blendshapes](https://mlsanity.com/paper/2605.01854/) (2026)
- [HandSCS: Structural Coordinate Space for Animatable Hand Gaussian Splatting](https://mlsanity.com/paper/2503.14736/) (2025)
- [Hyper Diffusion Avatars: Dynamic Human Avatar Generation using Network Weight Space Diffusion](https://mlsanity.com/paper/2509.04145/) (2025)
- [Large-Scale High-Quality 3D Gaussian Head Reconstruction from Multi-View Captures](https://mlsanity.com/paper/2605.04035/) (2026)
- [Physically Plausible Human-Object Rendering from Sparse Views via 3D Gaussian Splatting](https://mlsanity.com/paper/2503.09640/) (2025)
- [EgoAnimate: Generating Human Animations from Egocentric top-down Views](https://mlsanity.com/paper/2507.09230/) (2025)
- [MoGaFace: Momentum-Guided and Texture-Aware Gaussian Avatars for Consistent Facial Geometry](https://mlsanity.com/paper/2508.01218/) (2025)

## Abstract (arXiv)

Recent progress in neural rendering has brought forth pioneering methods, such as NeRF and Gaussian Splatting, which revolutionize view rendering across various domains like AR/VR, gaming, and content creation. While these methods excel at interpolating em within the training data, the challenge of generalizing to new scenes and objects from very sparse views persists. Specifically, modeling 3D humans from sparse views presents formidable hurdles due to the inherent complexity of human geometry, resulting in inaccurate reconstructions of geometry and textures. To tackle this challenge, this paper leverages recent advancements in Gaussian Splatting and introduces a new method to learn generalizable human Gaussians that allows photorealistic and accurate view-rendering of a new human subject from a limited set of sparse views in a feed-forward manner. A pivotal innovation of our approach involves reformulating the learning of 3D Gaussian parameters into a regression process defined on the 2D UV space of a human template, which allows leveraging the strong geometry prior and the advantages of 2D convolutions. In addition, a multi-scaffold is proposed to effectively represent the offset details. Our method outperforms recent methods on both within-dataset generalization as well as cross-dataset generalization settings.