mlsanityData to 14 Oct 2026

STCrowd: A Multimodal Dataset for Pedestrian Perception in Crowded Scenes

arXiv 2204.01026 ↗First posted 3 Apr 2022cs.CVcited by 35

Not on the map. Why?

Abstract (arXiv)

Accurately detecting and tracking pedestrians in 3D space is challenging due to large variations in rotations, poses and scales. The situation becomes even worse for dense crowds with severe occlusions. However, existing benchmarks either only provide 2D annotations, or have limited 3D annotations with low-density pedestrian distribution, making it difficult to build a reliable pedestrian perception system especially in crowded scenes. To better evaluate pedestrian perception algorithms in crowded scenarios, we introduce a large-scale multimodal dataset,STCrowd. Specifically, in STCrowd, there are a total of 219 K pedestrian instances and 20 persons per frame on average, with various levels of occlusion. We provide synchronized LiDAR point clouds and camera images as well as their corresponding 3D labels and joint IDs. STCrowd can be used for various tasks, including LiDAR-only, image-only, and sensor-fusion based pedestrian detection and tracking. We provide baselines for most of the tasks. In addition, considering the property of sparse global distribution and density-varying local distribution of pedestrians, we further propose a novel method, Density-aware Hierarchical heatmap Aggregation (DHA), to enhance pedestrian perception in crowded scenes. Extensive experiments show that our new method achieves state-of-the-art performance for pedestrian detection on various datasets.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 46 of its 64 references that are in the map. First posted on arXiv 3 Apr 2022.

Builds on

46 of 64 references.

References · 46

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  4. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  5. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  6. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,288
  7. 2019Objects as Pointscited by 1,175
  8. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  9. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 941
  10. 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 910
  11. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 732
  12. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 724
34 more
  1. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  2. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  3. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  4. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 389
  5. 2019Tracking without bells and whistlescited by 382
  6. 2019From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Networkcited by 368
  7. 2019PIXOR: Real-time 3D Object Detection from Point Cloudscited by 358
  8. 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
  9. 2017Joint 3D Proposal Generation and Object Detection from View Aggregationcited by 326
  10. 2015LSTM: A Search Space Odysseycited by 316
  11. 2018CrowdHuman: A Benchmark for Detecting Human in a Crowdcited by 284
  12. 2019STD: Sparse-to-Dense 3D Object Detector for Point Cloudcited by 282
  13. 2020Deep Continuous Fusion for Multi-Sensor 3D Object Detectioncited by 263
  14. 2020Multi-Task Multi-Sensor Fusion for 3D Object Detectioncited by 203
  15. 20203D-CVF: Generating Joint Camera and LiDAR Features Using Cross-View Spatial Feature Fusion for 3D Object Detectioncited by 176
  16. 2017CityPersons: A Diverse Dataset for Pedestrian Detectioncited by 172
  17. 2019End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Cloudscited by 171
  18. 2020ImVoteNet: Boosting 3D Object Detection in Point Clouds with Image Votescited by 119
  19. 2018TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agentscited by 111
  20. 2019Object as Hotspots: An Anchor-Free 3D Object Detection Approach via Firing of Hotspotscited by 84
  21. 2018TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactionscited by 82
  22. 2019The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenescited by 80
  23. 2018Joint Monocular 3D Vehicle Detection and Trackingcited by 77
  24. 2020SSN: Shape Signature Networks for Multi-class Object Detection from Point Cloudscited by 74
  25. 2020Probabilistic 3D Multi-Object Tracking for Autonomous Drivingcited by 64
  26. 2021Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perceptioncited by 57
  27. 2019PI-RCNN: An Efficient Multi-sensor 3D Object Detector with Point-based Attentive Cont-conv Fusion Modulecited by 56
  28. 20193D Multi-Object Tracking: A Baseline and New Evaluation Metricscited by 47
  29. 2019WiderPerson: A Diverse Dataset for Dense Pedestrian Detection in the Wildcited by 39
  30. 2019D2-City: A Large-Scale Dashcam Video Dataset of Diverse Traffic Scenarioscited by 28
  31. 2018PedX: Benchmark Dataset for Metric 3D Pose Estimation of Pedestrians in Complex Urban Intersectionscited by 27
  32. 2019StarNet: Pedestrian Trajectory Prediction using Deep Neural Network in Star Topologycited by 17
  33. 2020A3D: Adaptive 3D Networks for Video Action Recognitioncited by 5
  34. 2019DensePeds: Pedestrian Tracking in Dense Crowds Using Front-RVO and Sparse Featurescited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 6; 2023: 7; 2024: 12; 2025: 9; 2026: 1.

35 papers.

Sort:

By area:

  1. 2022Vision-Centric BEV Perception: A Surveycited by 126
  2. 2024A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlookcited by 66
  3. 2022LidarGait: Benchmarking 3D Gait Recognition with Point Cloudscited by 55
  4. 2022LiDAR-aid Inertial Poser: Large-scale Human Motion Capture by Sparse Inertial and LiDAR Sensorscited by 44
  5. 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 43
  6. 2022CL3D: Unsupervised Domain Adaptation for Cross-LiDAR 3D Detectioncited by 27
  7. 2022Weakly Supervised 3D Multi-person Pose Estimation for Large-scale Scenes based on Monocular Camera and Single LiDARcited by 23
  8. 2024LiveHPS: LiDAR-based Scene-level Human Pose and Shape Estimation in Free Environmentcited by 20
  9. 2023Human-centric Scene Understanding for 3D Large-scale Scenarioscited by 17
  10. 2022Gait Recognition in Large-scale Free Environment via Single LiDARcited by 16
  11. 2024RELI11D: A Comprehensive Multimodal Human Motion Dataset and Methodcited by 16
  12. 2025Towards Balanced Multi-Modal Learning in 3D Human Pose Estimationcited by 12
23 more
  1. 2024When Pedestrian Detection Meets Multi-Modal Learning: Generalist Model and Benchmark Datasetcited by 10
  2. 2023WildRefer: 3D Object Localization in Large-scale Dynamic Scenes with Multi-modal Visual Data and Natural Languagecited by 8
  3. 2023An Efficient Semi-Automated Scheme for Infrastructure LiDAR Annotationcited by 7
  4. 2024Visual Modality Prompt for Adapting Vision-Language Object Detectorscited by 7
  5. 2024LiveHPS++: Robust and Coherent Motion Capture in Dynamic Free Environmentcited by 6
  6. 2024Pre-training a Density-Aware Pose Transformer for Robust LiDAR-based 3D Human Pose Estimationcited by 5
  7. 2023One Training for Multiple Deployments: Polar-based Adaptive BEV Perception for Autonomous Drivingcited by 3
  8. 2023ContrastMotion: Self-supervised Scene Motion Learning for Large-Scale LiDAR Point Cloudscited by 3
  9. 2024FreeCap: Hybrid Calibration-Free Motion Capture in Open Environmentscited by 3
  10. 2024HUNTER: Unsupervised Human-centric 3D Detection via Transferring Knowledge from Synthetic Instances to Real Scenescited by 2
  11. 2025Learning better representations for crowded pedestrians in offboard LiDAR-camera 3D tracking-by-detectioncited by 2
  12. 2025SEPose: A Synthetic Event-based Human Pose Estimation Dataset for Pedestrian Monitoringcited by 2
  13. 2023HumanReg: Self-supervised Non-rigid Registration of Human Point Cloudcited by 1
  14. 2024TrajLearn: Trajectory Prediction Learning using Deep Generative Modelscited by 1
  15. 2025PFSD: A Multi-Modal Pedestrian-Focus Scene Dataset for Rich Tasks in Semi-Structured Environmentscited by 1
  16. 2025CrowdQuery: Density-Guided Query Module for Enhanced 2D and 3D Detection in Crowded Scenescited by 1
  17. 2023Semantic and Articulated Pedestrian Sensing Onboard a Moving Vehiclecited by 0
  18. 2024MCPDepth: Omnidirectional Depth Estimation via Stereo Matching from Multi-Cylindrical Panoramascited by 0
  19. 2025A Spatio-temporal Graph Network Allowing Incomplete Trajectory Input for Pedestrian Trajectory Predictioncited by 0
  20. 20253D Human Pose and Shape Estimation from LiDAR Point Clouds: A Reviewcited by 0
  21. 2025PCICF: A Pedestrian Crossing Identification and Classification Frameworkcited by 0
  22. 2025Large Language Models and 3D Vision for Intelligent Robotic Perception and Autonomycited by 0
  23. 2026MamMA: A Mamba-Based Pedestrian Trajectory Prediction Algorithm Considering Occupancy Map and Pedestrian Awareness Statescited by 0