mlsanityData to 7 Oct 2026

Weakly Supervised Multi-Modal 3D Human Body Pose Estimation for Autonomous Driving

arXiv 2307.14889 ↗First posted 27 Jul 2023cs.CV cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Accurate 3D human pose estimation (3D HPE) is crucial for enabling autonomous vehicles (AVs) to make informed decisions and respond proactively in critical road scenarios. Promising results of 3D HPE have been gained in several domains such as human-computer interaction, robotics, sports and medical analytics, often based on data collected in well-controlled laboratory environments. Nevertheless, the transfer of 3D HPE methods to AVs has received limited research attention, due to the challenges posed by obtaining accurate 3D pose annotations and the limited suitability of data from other domains. We present a simple yet efficient weakly supervised approach for 3D HPE in the AV context by employing a high-level sensor fusion between camera and LiDAR data. The weakly supervised setting enables training on the target datasets without any 2D/3D keypoint labels by using an off-the-shelf 2D joint extractor and pseudo labels generated from LiDAR to image projections. Our approach outperforms state-of-the-art results by up to ~ 13% on the Waymo Open Dataset in the weakly supervised setting and achieves state-of-the-art results in the supervised setting.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 28 of its 41 references that are in the map. First posted on arXiv 27 Jul 2023.

Builds on

28 of 41 references.

References · 28

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  5. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  6. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  7. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
  8. 20183D human pose estimation in video with temporal convolutions and semi-supervised trainingcited by 401
  9. 2017On human motion prediction using recurrent neural networkscited by 336
  10. 20213D Human Pose Estimation with Spatial and Temporal Transformerscited by 271
  11. 2019Learning Trajectory Dependencies for Human Motion Predictioncited by 245
  12. 2019Semantic Graph Convolutional Networks for 3D Human Pose Regressioncited by 204
16 more
  1. 2022AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Timecited by 199
  2. 2020Long-term Human Motion Prediction with Scene Contextcited by 191
  3. 2021MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimationcited by 151
  4. 2019Self-Supervised Learning of 3D Human Pose using Multi-view Geometrycited by 98
  5. 2021Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimationcited by 94
  6. 2021Multi-Person 3D Motion Prediction with Multi-Range Transformerscited by 63
  7. 2018The EuroCity Persons Dataset: A Novel Benchmark for Object Detectioncited by 56
  8. 2018Can 3D Pose be Learned from 2D Projections Alone?cited by 44
  9. 2022MotionMixer: MLP-based 3D Human Body Pose Forecastingcited by 42
  10. 2021Multi-modal 3D Human Pose Estimation with 2D Weak Supervision in Autonomous Drivingcited by 39
  11. 2018PedX: Benchmark Dataset for Metric 3D Pose Estimation of Pedestrians in Complex Urban Intersectionscited by 27
  12. 2020Traffic Control Gesture Recognition for Autonomous Vehiclescited by 24
  13. 2022HUM3DIL: Semi-supervised Multi-modal 3D Human Pose Estimation for Autonomous Drivingcited by 22
  14. 2021Self-Supervised 3D Human Pose Estimation with Multiple-View Geometrycited by 10
  15. 2022On-Board Pedestrian Trajectory Prediction Using Behavioral Featurescited by 5
  16. 2021Learning Temporal 3D Human Pose Estimation with Pseudo-Labelscited by 4

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 4.

8 papers.

  1. 2023MotionAGFormer: Enhancing 3D Human Pose Estimation with a Transformer-GCNFormer Networkcited by 62
  2. 2024Optimizing Local-Global Dependencies for Accurate 3D Human Pose Estimationcited by 3
  3. 2023Efficient, Self-Supervised Human Pose Estimation with Inductive Prior Tuningcited by 1
  4. 2024Towards a large-scale fused and labeled dataset of human pose while interacting with robots in shared urban areascited by 0
  5. 2025KASportsFormer: Kinematic Anatomy Enhanced Transformer for 3D Human Pose Estimation on Short Sports Scene Videocited by 0
  6. 2025VOccl3D: A Video Benchmark Dataset for 3D Human Pose and Shape Estimation under real Occlusionscited by 0
  7. 20253D Human Pose and Shape Estimation from LiDAR Point Clouds: A Reviewcited by 0
  8. 2025Point2Pose: A Generative Framework for 3D Human Pose Estimation with Multi-View Point Cloud Datasetcited by 0