# A2J: Anchor-to-Joint Regression Network for 3D Articulated Pose Estimation from a Single Depth Image

arXiv: https://arxiv.org/abs/1908.09999 · HTML: https://mlsanity.com/paper/1908.09999/

## Key facts

- Cited by 56 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 27 Aug 2019.

## Built on by (most cited first, 30 of 56)

- [DexYCB: A Benchmark for Capturing Hand Grasping of Objects](https://mlsanity.com/paper/2104.04631/) (2021)
- [Rethinking Counting and Localization in Crowds:A Purely Point-Based Framework](https://mlsanity.com/paper/2107.12746/) (2021)
- [Discrete Cosine Transform Network for Guided Depth Map Super-Resolution](https://mlsanity.com/paper/2104.06977/) (2021)
- [AlignSDF: Pose-Aligned Signed Distance Fields for Hand-Object Reconstruction](https://mlsanity.com/paper/2207.12909/) (2022)
- [gSDF: Geometry-Driven Signed Distance Functions for 3D Hand-Object Reconstruction](https://mlsanity.com/paper/2304.11970/) (2023)
- [Learning to Imitate Object Interactions from Internet Videos](https://mlsanity.com/paper/2211.13225/) (2022)
- [HUMAN4D: A Human-Centric Multimodal Dataset for Motions and Immersive Media](https://mlsanity.com/paper/2110.07235/) (2021)
- [Spherical Space Feature Decomposition for Guided Depth Map Super-Resolution](https://mlsanity.com/paper/2303.08942/) (2023)
- [EgoGen: An Egocentric Synthetic Data Generator](https://mlsanity.com/paper/2401.08739/) (2024)
- [Versatile Multi-Modal Pre-Training for Human-Centric Perception](https://mlsanity.com/paper/2203.13815/) (2022)
- [HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interaction](https://mlsanity.com/paper/2406.06843/) (2024)
- [DECA: Deep viewpoint-Equivariant human pose estimation using Capsule Autoencoders](https://mlsanity.com/paper/2108.08557/) (2021)
- [Deformer: Dynamic Fusion Transformer for Robust Hand Pose Estimation](https://mlsanity.com/paper/2303.04991/) (2023)
- [Efficient Annotation and Learning for 3D Hand Pose Estimation: A Survey](https://mlsanity.com/paper/2206.02257/) (2022)
- [Tracking and Reconstructing Hand Object Interactions from Point Cloud Sequences in the Wild](https://mlsanity.com/paper/2209.12009/) (2022)
- [A2J-Transformer: Anchor-to-Joint Transformer Network for 3D Interacting Hand Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/2304.03635/) (2023)
- [HOISDF: Constraining 3D Hand-Object Pose Estimation with Global Signed Distance Fields](https://mlsanity.com/paper/2402.17062/) (2024)
- [NeuralGrasps: Learning Implicit Representations for Grasps of Multiple Robotic Hands](https://mlsanity.com/paper/2207.02959/) (2022)
- [LiDAR-HMR: 3D Human Mesh Recovery from LiDAR](https://mlsanity.com/paper/2311.11971/) (2023)
- [HandDiff: 3D Hand Pose Estimation with Diffusion on Image-Point Cloud](https://mlsanity.com/paper/2404.03159/) (2024)
- [HandVoxNet++: 3D Hand Shape and Pose Estimation using Voxel-Based Neural Networks](https://mlsanity.com/paper/2107.01205/) (2021)
- [HORT: Monocular Hand-held Objects Reconstruction with Transformers](https://mlsanity.com/paper/2503.21313/) (2025)
- [LPFormer: LiDAR Pose Estimation Transformer with Multi-Task Network](https://mlsanity.com/paper/2306.12525/) (2023)
- [PointHPS: Cascaded 3D Human Pose and Shape Estimation from Point Clouds](https://mlsanity.com/paper/2308.14492/) (2023)
- [HandOS: 3D Hand Reconstruction in One Stage](https://mlsanity.com/paper/2412.01537/) (2024)
- [The Best of Both Worlds: Combining Model-based and Nonparametric Approaches for 3D Human Body Estimation](https://mlsanity.com/paper/2205.00508/) (2022)
- [TriHorn-Net: A Model for Accurate Depth-Based 3D Hand Pose Estimation](https://mlsanity.com/paper/2206.07117/) (2022)
- [SiMHand: Mining Similar Hands for Large-Scale 3D Hand Pose Pre-training](https://mlsanity.com/paper/2502.15251/) (2025)
- [Local and Global Point Cloud Reconstruction for 3D Hand Pose Estimation](https://mlsanity.com/paper/2112.06389/) (2021)
- [Efficient Virtual View Selection for 3D Hand Pose Estimation](https://mlsanity.com/paper/2203.15458/) (2022)

## Abstract (arXiv)

For 3D hand and body pose estimation task in depth image, a novel anchor-based approach termed Anchor-to-Joint regression network (A2J) with the end-to-end learning ability is proposed. Within A2J, anchor points able to capture global-local spatial context information are densely set on depth image as local regressors for the joints. They contribute to predict the positions of the joints in ensemble way to enhance generalization ability. The proposed 3D articulated pose estimation paradigm is different from the state-of-the-art encoder-decoder based FCN, 3D CNN and point-set based manners. To discover informative anchor points towards certain joint, anchor proposal procedure is also proposed for A2J. Meanwhile 2D CNN (i.e., ResNet-50) is used as backbone network to drive A2J, without using time-consuming 3D convolutional or deconvolutional layers. The experiments on 3 hand datasets and 2 body datasets verify A2J's superiority. Meanwhile, A2J is of high running speed around 100 FPS on single NVIDIA 1080Ti GPU.