# 3D Human Keypoints Estimation From Point Clouds in the Wild Without Human Labels

arXiv: https://arxiv.org/abs/2306.04745 · HTML: https://mlsanity.com/paper/2306.04745/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 37 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling](https://mlsanity.com/paper/2111.14819/) (2021)
- [PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding](https://mlsanity.com/paper/2007.10985/) (2020)
- [Self-Supervised Pretraining of 3D Features on any Point-Cloud](https://mlsanity.com/paper/2101.02691/) (2021)
- [Unsupervised Point Cloud Pre-Training via Occlusion Completion](https://mlsanity.com/paper/2010.01089/) (2020)
- [Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds](https://mlsanity.com/paper/2109.00179/) (2021)
- [Unsupervised Learning of Object Landmarks through Conditional Image Generation](https://mlsanity.com/paper/1806.07823/) (2018)
- [Neural Scene Flow Prior](https://mlsanity.com/paper/2111.01253/) (2021)
- [Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning](https://mlsanity.com/paper/1807.03146/) (2018)
- [USIP: Unsupervised Stable Interest Point Detection from 3D Point Clouds](https://mlsanity.com/paper/1904.00229/) (2019)
- [Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objects](https://mlsanity.com/paper/2112.11347/) (2021)
- [Multi-modal 3D Human Pose Estimation with 2D Weak Supervision in Autonomous Driving](https://mlsanity.com/paper/2112.12141/) (2021)
- [CASA: Category-agnostic Skeletal Animal Reconstruction](https://mlsanity.com/paper/2211.03568/) (2022)
- [LiDARCap: Long-range Marker-less 3D Human Motion Capture with LiDAR Point Clouds](https://mlsanity.com/paper/2203.14698/) (2022)
- [PedX: Benchmark Dataset for Metric 3D Pose Estimation of Pedestrians in Complex Urban Intersections](https://mlsanity.com/paper/1809.03605/) (2018)
- [Unsupervised Human Pose Estimation through Transforming Shape Templates](https://mlsanity.com/paper/2105.04154/) (2021)
- [Self-Supervised Keypoint Discovery in Behavioral Videos](https://mlsanity.com/paper/2112.05121/) (2021)
- [HUM3DIL: Semi-supervised Multi-modal 3D Human Pose Estimation for Autonomous Driving](https://mlsanity.com/paper/2212.07729/) (2022)
- [HPERL: 3D Human Pose Estimation from RGB and LiDAR](https://mlsanity.com/paper/2010.08221/) (2020)
- [UKPGAN: A General Self-Supervised Keypoint Detector](https://mlsanity.com/paper/2011.11974/) (2020)
- [Learning to Estimate 3D Human Pose from Point Cloud](https://mlsanity.com/paper/2212.12910/) (2022)

## Built on by (most cited first, 9 of 9)

- [LiDAR-HMR: 3D Human Mesh Recovery from LiDAR](https://mlsanity.com/paper/2311.11971/) (2023)
- [Pre-training a Density-Aware Pose Transformer for Robust LiDAR-based 3D Human Pose Estimation](https://mlsanity.com/paper/2412.13454/) (2024)
- [XR-MBT: Multi-modal Full Body Tracking for XR through Self-Supervision with Learned Depth Point Cloud Registration](https://mlsanity.com/paper/2411.18377/) (2024)
- [SPiKE: 3D Human Pose from Point Cloud Sequences](https://mlsanity.com/paper/2409.01879/) (2024)
- [Application of Vision-Language Model to Pedestrians Behavior and Scene Understanding in Autonomous Driving](https://mlsanity.com/paper/2501.06680/) (2025)
- [VoxelKP: A Voxel-based Network Architecture for Human Keypoint Estimation in LiDAR Data](https://mlsanity.com/paper/2312.08871/) (2023)
- [SelfGeo: Self-supervised and Geodesic-consistent Estimation of Keypoints on Deformable Shapes](https://mlsanity.com/paper/2408.02291/) (2024)
- [3D Human Pose and Shape Estimation from LiDAR Point Clouds: A Review](https://mlsanity.com/paper/2509.12197/) (2025)
- [Learning Human-Object Interaction for 3D Human Pose Estimation from LiDAR Point Clouds](https://mlsanity.com/paper/2603.16343/) (2026)

## Abstract (arXiv)

Training a 3D human keypoint detector from point clouds in a supervised manner requires large volumes of high quality labels. While it is relatively easy to capture large amounts of human point clouds, annotating 3D keypoints is expensive, subjective, error prone and especially difficult for long-tail cases (pedestrians with rare poses, scooterists, etc.). In this work, we propose GC-KPL - Geometry Consistency inspired Key Point Leaning, an approach for learning 3D human joint locations from point clouds without human labels. We achieve this by our novel unsupervised loss formulations that account for the structure and movement of the human body. We show that by training on a large training set from Waymo Open Dataset without any human annotated keypoints, we are able to achieve reasonable performance as compared to the fully supervised approach. Further, the backbone benefits from the unsupervised training and is useful in downstream fewshot learning of keypoints, where fine-tuning on only 10 percent of the labeled training data gives comparable performance to fine-tuning on the entire set. We demonstrated that GC-KPL outperforms by a large margin over SoTA when trained on entire dataset and efficiently leverages large volumes of unlabeled data.