# 3D Points Splatting for Real-Time Dynamic Hand Reconstruction

arXiv: https://arxiv.org/abs/2312.13770 · HTML: https://mlsanity.com/paper/2312.13770/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 39 references that are in the map.
- First posted on arXiv 21 Dec 2023.

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [Point-NeRF: Point-based Neural Radiance Fields](https://mlsanity.com/paper/2201.08845/) (2022)
- [HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video](https://mlsanity.com/paper/2201.04127/) (2022)
- [SynSin: End-to-end View Synthesis from a Single Image](https://mlsanity.com/paper/1912.08804/) (2019)
- [InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/2008.09309/) (2020)
- [NeuMan: Neural Human Radiance Field from a Single Video](https://mlsanity.com/paper/2203.12575/) (2022)
- [Differentiable Surface Splatting for Point-based Geometry Processing](https://mlsanity.com/paper/1906.04173/) (2019)
- [GRAM: Generative Radiance Manifolds for 3D-Aware Image Generation](https://mlsanity.com/paper/2112.08867/) (2021)
- [Neural Head Avatars from Monocular RGB Videos](https://mlsanity.com/paper/2112.01554/) (2021)
- [PointAvatar: Deformable Point-based Head Avatars from Videos](https://mlsanity.com/paper/2212.08377/) (2022)
- [Grasping Field: Learning Implicit Representations for Human Grasps](https://mlsanity.com/paper/2008.04451/) (2020)
- [SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Video](https://mlsanity.com/paper/2201.12792/) (2022)
- [LEAP: Learning Articulated Occupancy of People](https://mlsanity.com/paper/2104.06849/) (2021)
- [imGHUM: Implicit Generative Models of 3D Human Shape and Articulated Pose](https://mlsanity.com/paper/2108.10842/) (2021)
- [Neural Point Catacaustics for Novel-View Synthesis of Reflections](https://mlsanity.com/paper/2301.01087/) (2023)
- [Model-based 3D Hand Reconstruction via Self-Supervised Learning](https://mlsanity.com/paper/2103.11703/) (2021)
- [ImFace: A Nonlinear 3D Morphable Face Model with Implicit Neural Representations](https://mlsanity.com/paper/2203.14510/) (2022)
- [LISA: Learning Implicit Shape and Appearance of Hands](https://mlsanity.com/paper/2204.01695/) (2022)
- [Hand Avatar: Free-Pose Hand Animation and Rendering from Monocular Video](https://mlsanity.com/paper/2211.12782/) (2022)
- [A Probabilistic Attention Model with Occlusion-aware Texture Regression for 3D Hand Reconstruction from a Single RGB Image](https://mlsanity.com/paper/2304.14299/) (2023)
- [HARP: Personalized Hand Reconstruction from a Monocular RGB Video](https://mlsanity.com/paper/2212.09530/) (2022)
- [HandNeRF: Neural Radiance Fields for Animatable Interacting Hands](https://mlsanity.com/paper/2303.13825/) (2023)
- [Dynamic Mesh-Aware Radiance Fields](https://mlsanity.com/paper/2309.04581/) (2023)

## Built on by (most cited first, 10 of 10)

- [Recent Advances in 3D Gaussian Splatting](https://mlsanity.com/paper/2403.11134/) (2024)
- [3D Gaussian as a New Era: A Survey](https://mlsanity.com/paper/2402.07181/) (2024)
- [ArticulatedGS: Self-supervised Digital Twin Modeling of Articulated Objects using 3D Gaussian Splatting](https://mlsanity.com/paper/2503.08135/) (2025)
- [Learning Coherent Matrixized Representation in Latent Space for Volumetric 4D Generation](https://mlsanity.com/paper/2403.13238/) (2024)
- [Generalizable Neural Human Renderer](https://mlsanity.com/paper/2404.14199/) (2024)
- [Learning Interaction-aware 3D Gaussian Splatting for One-shot Hand Avatars](https://mlsanity.com/paper/2410.08840/) (2024)
- [JGHand: Joint-Driven Animatable Hand Avater via 3D Gaussian Splatting](https://mlsanity.com/paper/2501.19088/) (2025)
- [Relightable Holoported Characters: Capturing and Relighting Dynamic Human Performance from Sparse Views](https://mlsanity.com/paper/2512.00255/) (2025)
- [HandSCS: Structural Coordinate Space for Animatable Hand Gaussian Splatting](https://mlsanity.com/paper/2503.14736/) (2025)
- [MASS: Mesh-inellipse Aligned Deformable Surfel Splatting for Hand Reconstruction and Rendering from Egocentric Monocular Video](https://mlsanity.com/paper/2604.08943/) (2026)

## Abstract (arXiv)

We present 3D Points Splatting Hand Reconstruction (3D-PSHR), a real-time and photo-realistic hand reconstruction approach. We propose a self-adaptive canonical points upsampling strategy to achieve high-resolution hand geometry representation. This is followed by a self-adaptive deformation that deforms the hand from the canonical space to the target pose, adapting to the dynamic changing of canonical points which, in contrast to the common practice of subdividing the MANO model, offers greater flexibility and results in improved geometry fitting. To model texture, we disentangle the appearance color into the intrinsic albedo and pose-aware shading, which are learned through a Context-Attention module. Moreover, our approach allows the geometric and the appearance models to be trained simultaneously in an end-to-end manner. We demonstrate that our method is capable of producing animatable, photorealistic and relightable hand reconstructions using multiple datasets, including monocular videos captured with handheld smartphones and large-scale multi-view videos featuring various hand poses. We also demonstrate that our approach achieves real-time rendering speeds while simultaneously maintaining superior performance compared to existing state-of-the-art methods.