mlsanityData to 7 Oct 2026

Weakly Supervised 3D Multi-person Pose Estimation for Large-scale Scenes based on Monocular Camera and Single LiDAR

arXiv 2211.16951 ↗First posted 30 Nov 2022cs.CVcited by 23

Not on the map. Why?

Abstract (arXiv)

Depth estimation is usually ill-posed and ambiguous for monocular camera-based 3D multi-person pose estimation. Since LiDAR can capture accurate depth information in long-range scenes, it can benefit both the global localization of individuals and the 3D pose estimation by providing rich geometry features. Motivated by this, we propose a monocular camera and single LiDAR-based method for 3D multi-person pose estimation in large-scale scenes, which is easy to deploy and insensitive to light. Specifically, we design an effective fusion strategy to take advantage of multi-modal input data, including images and point cloud, and make full use of temporal information to guide the network to learn natural and coherent human motions. Without relying on any 3D pose annotations, our method exploits the inherent geometry constraints of point cloud for self-supervision and utilizes 2D keypoints on images for weak supervision. Extensive experiments on public datasets and our newly collected dataset demonstrate the superiority and generalization capability of our proposed method.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 48 of its 58 references that are in the map. First posted on arXiv 30 Nov 2022.

Builds on

48 of 58 references.

References · 48

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  3. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
  4. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  5. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  6. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  7. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  8. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
  9. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 493
  10. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 464
  11. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  12. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
36 more
  1. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
  2. 2017Joint 3D Proposal Generation and Object Detection from View Aggregationcited by 325
  3. 2020HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimationcited by 304
  4. 20213D Human Pose Estimation with Spatial and Temporal Transformerscited by 271
  5. 2020Deep Continuous Fusion for Multi-Sensor 3D Object Detectioncited by 263
  6. 2016Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Posecited by 220
  7. 2020Monocular, One-stage, Regression of Multiple 3D Peoplecited by 216
  8. 2017Single-Shot Multi-Person 3D Pose Estimation From Monocular RGBcited by 195
  9. 2022MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Videocited by 158
  10. 2021Putting People in their Place: Monocular Regression of 3D People in Depthcited by 141
  11. 2019Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Imagecited by 131
  12. 2019Exploiting temporal context for 3D human pose estimation in the wildcited by 110
  13. 2019Self-Supervised Learning of 3D Human Pose using Multi-view Geometrycited by 98
  14. 2019Unsupervised 3D Pose Estimation with Geometric Self-Supervisioncited by 95
  15. 2018LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Imagescited by 88
  16. 2019Fast and Robust Multi-Person 3D Pose Estimation from Multiple Viewscited by 83
  17. 2020SSN: Shape Signature Networks for Multi-class Object Detection from Point Cloudscited by 74
  18. 2017Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Imagecited by 73
  19. 2018Unsupervised Geometry-Aware Representation for 3D Human Pose Estimationcited by 73
  20. 2020CanonPose: Self-Supervised Monocular 3D Human Pose Estimation in the Wildcited by 65
  21. 2021Direct Multi-view Multi-person 3D Pose Estimationcited by 64
  22. 2021Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perceptioncited by 57
  23. 20203D Human Pose Estimation using Spatio-Temporal Networks with Explicit Occlusion Trainingcited by 54
  24. 2020SMAP: Single-Shot Multi-Person Absolute 3D Pose Estimationcited by 52
  25. 2020HMOR: Hierarchical Multi-Person Ordinal Relations for Monocular Multi-Person 3D Pose Estimationcited by 44
  26. 20214D-Net for Learned Multi-Modal Alignmentcited by 44
  27. 2022LiDAR-aid Inertial Poser: Large-scale Human Motion Capture by Sparse Inertial and LiDAR Sensorscited by 43
  28. 2021Multi-modal 3D Human Pose Estimation with 2D Weak Supervision in Autonomous Drivingcited by 39
  29. 2022STCrowd: A Multimodal Dataset for Pedestrian Perception in Crowded Scenescited by 35
  30. 20183D Human Pose Estimation in RGBD Images for Robotic Task Learningcited by 33
  31. 2022LiDARCap: Long-range Marker-less 3D Human Motion Capture with LiDAR Point Cloudscited by 32
  32. 2022HybridCap: Inertia-aid Monocular Capture of Challenging Human Motionscited by 19
  33. 2020HPERL: 3D Human Pose Estimation from RGB and LiDARcited by 18
  34. 2019Absolute Human Pose Estimation with Depth Prediction Networkcited by 9
  35. 2021Input-Output Balanced Framework for Long-tailed LiDAR Semantic Segmentationcited by 8
  36. 2022NIMBLE: A Non-rigid Hand Model with Bones and Musclescited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 10; 2025: 5; 2026: 1.

23 papers.

Sort:

By area:

  1. 2023MM-Fi: Multi-Modal Non-Intrusive 4D Human Dataset for Versatile Wireless Sensingcited by 77
  2. 2023Learning to Adapt SAM for Segmenting Cross-domain Point Cloudscited by 20
  3. 2024RELI11D: A Comprehensive Multimodal Human Motion Dataset and Methodcited by 16
  4. 2024A Unified Diffusion Framework for Scene-aware Human Motion Estimation from Sparse Signalscited by 15
  5. 2023LiDAR-HMR: 3D Human Mesh Recovery from LiDARcited by 12
  6. 2024DriveDiTFit: Fine-tuning Diffusion Transformers for Autonomous Drivingcited by 11
  7. 2023LPFormer: LiDAR Pose Estimation Transformer with Multi-Task Networkcited by 9
  8. 2023WildRefer: 3D Object Localization in Large-scale Dynamic Scenes with Multi-modal Visual Data and Natural Languagecited by 8
  9. 2023MOVIN: Real-time Motion Capture using a Single LiDARcited by 6
  10. 2025ViBES: A Conversational Agent with Behaviorally-Intelligent 3D Virtual Bodycited by 6
  11. 2024Pre-training a Density-Aware Pose Transformer for Robust LiDAR-based 3D Human Pose Estimationcited by 4
  12. 2024A Unified Framework for Human-centric Point Cloud Video Understandingcited by 3
11 more
  1. 2024FreeCap: Hybrid Calibration-Free Motion Capture in Open Environmentscited by 3
  2. 2025ClimbingCap: Multi-Modal Dataset and Method for Rock Climbing in World Coordinatecited by 3
  3. 2024HUNTER: Unsupervised Human-centric 3D Detection via Transferring Knowledge from Synthetic Instances to Real Scenescited by 2
  4. 2024Sports Analysis and VR Viewing System Based on Player Tracking and Pose Estimation with Multimodal and Multiview Sensorscited by 2
  5. 2024Towards Practical Human Motion Prediction with LiDAR Point Cloudscited by 2
  6. 2023LiCamPose: Combining Multi-View LiDAR and RGB Cameras for Robust Single-timestamp 3D Human Pose Estimationcited by 1
  7. 2024ELMO: Enhanced Real-time LiDAR Motion Capture through Upsamplingcited by 1
  8. 2025Sixth-Sense: Self-Supervised Learning of Spatial Awareness of Humans from a Planar Lidarcited by 0
  9. 2025Domain Adaptation-Based Crossmodal Knowledge Distillation for 3D Semantic Segmentationcited by 0
  10. 20253D Human Pose and Shape Estimation from LiDAR Point Clouds: A Reviewcited by 0
  11. 2026Learning Human-Object Interaction for 3D Human Pose Estimation from LiDAR Point Cloudscited by 0