mlsanityData to 14 Oct 2026

HUM3DIL: Semi-supervised Multi-modal 3D Human Pose Estimation for Autonomous Driving

arXiv 2212.07729 ↗First posted 15 Dec 2022cs.CVcited by 22

Not on the map. Why?

Abstract (arXiv)

Autonomous driving is an exciting new industry, posing important research questions. Within the perception module, 3D human pose estimation is an emerging technology, which can enable the autonomous vehicle to perceive and understand the subtle and complex behaviors of pedestrians. While hardware systems and sensors have dramatically improved over the decades -- with cars potentially boasting complex LiDAR and vision systems and with a growing expansion of the available body of dedicated datasets for this newly available information -- not much work has been done to harness these novel signals for the core problem of 3D human pose estimation. Our method, which we coin HUM3DIL (HUMan 3D from Images and LiDAR), efficiently makes use of these complementary signals, in a semi-supervised fashion and outperforms existing methods with a large margin. It is a fast and compact model for onboard deployment. Specifically, we embed LiDAR points into pixel-aligned multi-modal features, which we pass through a sequence of Transformer refinement stages. Quantitative experiments on the Waymo Open Dataset support these claims, where we achieve state-of-the-art results on the task of 3D pose estimation.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 34 of its 47 references that are in the map. First posted on arXiv 15 Dec 2022.

Builds on

34 of 47 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  4. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  5. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,200
  6. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  7. 2020PCT: Point cloud transformercited by 732
  8. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 627
  9. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 620
  10. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 418
  11. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  12. 2017Integral Human Pose Regressioncited by 256
22 more
  1. 2020Deep Learning-Based Human Pose Estimation: A Surveycited by 207
  2. 2021Offboard 3D Object Detection from Point Cloud Sequencescited by 145
  3. 2018BodyNet: Volumetric Inference of 3D Human Body Shapescited by 141
  4. 2020Coherent Reconstruction of Multiple Humans from a Single Imagecited by 136
  5. 2019Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Imagecited by 132
  6. 2019Exploiting temporal context for 3D human pose estimation in the wildcited by 110
  7. 2022Recovering 3D Human Mesh from Monocular Images: A Surveycited by 103
  8. 2017V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Mapcited by 102
  9. 2019DenseRaC: Joint 3D Pose and Shape Estimation by Dense Render-and-Comparecited by 100
  10. 2022Multi-modal Sensor Fusion for Auto Driving Perception: A Surveycited by 76
  11. 2020Hierarchical Kinematic Human Mesh Recoverycited by 71
  12. 2020Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wildcited by 66
  13. 20203D Human Mesh Regression with Dense Correspondencecited by 66
  14. 2020Monocular Human Pose Estimation: A Survey of Deep Learning-based Methodscited by 59
  15. 20203D Multi-bodies: Fitting Sets of Plausible 3D Human Models to Ambiguous Image Datacited by 50
  16. 2021THUNDR: Transformer-based 3D HUmaN Reconstruction with Markerscited by 50
  17. 2021Multi-modal 3D Human Pose Estimation with 2D Weak Supervision in Autonomous Drivingcited by 40
  18. 2020PointNet++ Grasping: Learning An End-to-end Spatial Grasp Generation Algorithm from Sparse Point Cloudscited by 39
  19. 2021Context Modeling in 3D Human Pose Estimation: A Unified Perspectivecited by 37
  20. 20183D Human Pose Estimation in RGBD Images for Robotic Task Learningcited by 33
  21. 2018PedX: Benchmark Dataset for Metric 3D Pose Estimation of Pedestrians in Complex Urban Intersectionscited by 27
  22. 2020HPERL: 3D Human Pose Estimation from RGB and LiDARcited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 8; 2025: 5; 2026: 1.

22 papers.

Sort:

By area:

  1. 2023Robots That Can See: Leveraging Human Pose for Trajectory Predictioncited by 23
  2. 2024FinePOSE: Fine-Grained Prompt-Driven 3D Human Pose Estimation via Diffusion Modelscited by 23
  3. 2023LiDAR-HMR: 3D Human Mesh Recovery from LiDARcited by 12
  4. 20233D Human Keypoints Estimation From Point Clouds in the Wild Without Human Labelscited by 9
  5. 2023LPFormer: LiDAR Pose Estimation Transformer with Multi-Task Networkcited by 9
  6. 2023Weakly Supervised Multi-Modal 3D Human Body Pose Estimation for Autonomous Drivingcited by 8
  7. 2024Human Modelling and Pose Estimation Overviewcited by 4
  8. 2024HFGaussian: Learning Generalizable Gaussian Human with Integrated Human Featurescited by 3
  9. 2024Di2Pose: Discrete Diffusion Model for Occluded 3D Human Pose Estimationcited by 2
  10. 2024Towards Multi-Modal Animal Pose Estimation: A Survey and In-Depth Analysiscited by 2
  11. 2023Efficient, Self-Supervised Human Pose Estimation with Inductive Prior Tuningcited by 1
  12. 2023VoxelKP: A Voxel-based Network Architecture for Human Keypoint Estimation in LiDAR Datacited by 1
10 more
  1. 2024Multi-hop graph transformer network for 3D human pose estimationcited by 1
  2. 2024SelfGeo: Self-supervised and Geodesic-consistent Estimation of Keypoints on Deformable Shapescited by 1
  3. 2025Robust Low-Light Human Pose Estimation through Illumination-Texture Modulationcited by 1
  4. 2025StarPose: 3D Human Pose Estimation via Spatial-Temporal Autoregressive Diffusioncited by 1
  5. 2023Semantic and Articulated Pedestrian Sensing Onboard a Moving Vehiclecited by 0
  6. 2024DOR3D-Net: Dense Ordinal Regression Network for 3D Hand Pose Estimationcited by 0
  7. 2025VOccl3D: A Video Benchmark Dataset for 3D Human Pose and Shape Estimation under real Occlusionscited by 0
  8. 20253D Human Pose and Shape Estimation from LiDAR Point Clouds: A Reviewcited by 0
  9. 2025Point2Pose: A Generative Framework for 3D Human Pose Estimation with Multi-View Point Cloud Datasetcited by 0
  10. 2026Learning Human-Object Interaction for 3D Human Pose Estimation from LiDAR Point Cloudscited by 0