# Point-to-Pose Voting based Hand Pose Estimation using Residual Permutation Equivariant Layer

arXiv: https://arxiv.org/abs/1812.02050 · HTML: https://mlsanity.com/paper/1812.02050/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Dec 2018.

## Built on by (most cited first, 21 of 21)

- [Towards Accurate Alignment in Real-time 3D Hand-Mesh Reconstruction](https://mlsanity.com/paper/2109.01723/) (2021)
- [LiDARCap: Long-range Marker-less 3D Human Motion Capture with LiDAR Point Clouds](https://mlsanity.com/paper/2203.14698/) (2022)
- [Hand Image Understanding via Deep Multi-Task Learning](https://mlsanity.com/paper/2107.11646/) (2021)
- [RGB2Hands: Real-Time Tracking of 3D Hand Interactions from Monocular RGB Video](https://mlsanity.com/paper/2106.11725/) (2021)
- [Tracking and Reconstructing Hand Object Interactions from Point Cloud Sequences in the Wild](https://mlsanity.com/paper/2209.12009/) (2022)
- [3D Pose Estimation of Two Interacting Hands from a Monocular Event Camera](https://mlsanity.com/paper/2312.14157/) (2023)
- [HandFoldingNet: A 3D Hand Pose Estimation Network Using Multiscale-Feature Guided Folding of a 2D Hand Skeleton](https://mlsanity.com/paper/2108.05545/) (2021)
- [HOISDF: Constraining 3D Hand-Object Pose Estimation with Global Signed Distance Fields](https://mlsanity.com/paper/2402.17062/) (2024)
- [HandDiff: 3D Hand Pose Estimation with Diffusion on Image-Point Cloud](https://mlsanity.com/paper/2404.03159/) (2024)
- [Anatomy-guided domain adaptation for 3D in-bed human pose estimation](https://mlsanity.com/paper/2211.12193/) (2022)
- [TriHorn-Net: A Model for Accurate Depth-Based 3D Hand Pose Estimation](https://mlsanity.com/paper/2206.07117/) (2022)
- [Local and Global Point Cloud Reconstruction for 3D Hand Pose Estimation](https://mlsanity.com/paper/2112.06389/) (2021)
- [Neural Voting Field for Camera-Space 3D Hand Pose Estimation](https://mlsanity.com/paper/2305.04328/) (2023)
- [Generalized Product-of-Experts for Learning Multimodal Representations in Noisy Environments](https://mlsanity.com/paper/2211.03587/) (2022)
- [A hybrid classification-regression approach for 3D hand pose estimation using graph convolutional networks](https://mlsanity.com/paper/2105.10902/) (2021)
- [Pyramid Deep Fusion Network for Two-Hand Reconstruction from RGB-D Images](https://mlsanity.com/paper/2307.06038/) (2023)
- [Skeleton-aware multi-scale heatmap regression for 2D hand pose estimation](https://mlsanity.com/paper/2105.10904/) (2021)
- [Pushing the Envelope for Depth-Based Semi-Supervised 3D Hand Pose Estimation with Consistency Training](https://mlsanity.com/paper/2303.15147/) (2023)
- [HandDAGT: A Denoising Adaptive Graph Transformer for 3D Hand Pose Estimation](https://mlsanity.com/paper/2407.20542/) (2024)
- [Rotation Equivariant 3D Hand Mesh Generation from a Single RGB Image](https://mlsanity.com/paper/2111.13023/) (2021)
- [HandMCM: Multi-modal Point Cloud-based Correspondence State Space Model for 3D Hand Pose Estimation](https://mlsanity.com/paper/2602.01586/) (2026)

## Abstract (arXiv)

Recently, 3D input data based hand pose estimation methods have shown state-of-the-art performance, because 3D data capture more spatial information than the depth image. Whereas 3D voxel-based methods need a large amount of memory, PointNet based methods need tedious preprocessing steps such as K-nearest neighbour search for each point. In this paper, we present a novel deep learning hand pose estimation method for an unordered point cloud. Our method takes 1024 3D points as input and does not require additional information. We use Permutation Equivariant Layer (PEL) as the basic element, where a residual network version of PEL is proposed for the hand pose estimation task. Furthermore, we propose a voting based scheme to merge information from individual points to the final pose output. In addition to the pose estimation task, the voting-based scheme can also provide point cloud segmentation result without ground-truth for segmentation. We evaluate our method on both NYU dataset and the Hands2017Challenge dataset. Our method outperforms recent state-of-the-art methods, where our pose accuracy is currently the best for the Hands2017Challenge dataset.