mlsanityData to 14 Oct 2026

TBD Pedestrian Data Collection: Towards Rich, Portable, and Large-Scale Natural Pedestrian Data

arXiv 2309.17187 ↗First posted 29 Sep 2023cs.CV cs.HC cs.ROcited by 6

Not on the map. Why?

Abstract (arXiv)

Social navigation and pedestrian behavior research has shifted towards machine learning-based methods and converged on the topic of modeling inter-pedestrian interactions and pedestrian-robot interactions. For this, large-scale datasets that contain rich information are needed. We describe a portable data collection system, coupled with a semi-autonomous labeling pipeline. As part of the pipeline, we designed a label correction web app that facilitates human verification of automated pedestrian tracking outcomes. Our system enables large-scale data collection in diverse environments and fast trajectory label production. Compared with existing pedestrian data collection methods, our system contains three components: a combination of top-down and ego-centric views, natural human behavior in the presence of a socially appropriate "robot", and human-verified labels grounded in the metric space. To the best of our knowledge, no prior data collection system has a combination of all three components. We further introduce our ever-expanding dataset from the ongoing data collection effort -- the TBD Pedestrian Dataset and show that our collected data is larger in scale, contains richer information when compared to prior datasets with human-verified labels, and supports new research opportunities.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 26 of its 51 references that are in the map. First posted on arXiv 29 Sep 2023.

Builds on

26 of 51 references.

References · 26

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  2. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  3. 2021ByteTrack: Multi-Object Tracking by Associating Every Detection Boxcited by 823
  4. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 707
  5. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 690
  6. 2020MOT20: A benchmark for multi object tracking in crowded scenescited by 357
  7. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 331
  8. 2020Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Predictioncited by 325
  9. 2018SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraintscited by 298
  10. 2019Human Motion Trajectory Prediction: A Surveycited by 232
  11. 2018Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learningcited by 175
  12. 2020From Goals, Waypoints & Paths To Long Term Human Trajectory Forecastingcited by 167
14 more
  1. 2022Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigationcited by 127
  2. 2018Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learningcited by 111
  3. 2021Core Challenges of Social Robot Navigation: A Surveycited by 106
  4. 2019The Garden of Forking Paths: Towards Multi-Future Trajectory Predictioncited by 72
  5. 2017Socially Compliant Navigation through Raw Depth Inputs with Generative Adversarial Imitation Learningcited by 52
  6. 2019TH"OR: Human-Robot Navigation Data Collection and Accurate Motion Trajectories Datasetcited by 36
  7. 2019Robot Navigation in Crowds by Graph Convolutional Networks with Attention Learned from Human Gazecited by 30
  8. 2021SocNavBench: A Grounded Simulation Testing Framework for Evaluating Social Navigationcited by 24
  9. 2020Risk-Sensitive Sequential Action Control with Multi-Modal Human Trajectory Forecasting for Safe Crowd-Robot Interactioncited by 21
  10. 2020SEAN: Social Environment for Autonomous Navigationcited by 18
  11. 2021Group-based Motion Prediction for Navigation in Crowded Environmentscited by 15
  12. 2019Dynamic Channel: A Planning Framework for Crowd Navigationcited by 10
  13. 2021Move Beyond Trajectories: Distribution Space Coupling for Crowd Navigationcited by 10
  14. 2022Pedestrian-Robot Interactions on Autonomous Crowd Navigation: Reactive Control Methods and Evaluation Metricscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

  1. 2025EgoTraj-Bench: Towards Robust Trajectory Prediction Under Ego-view Noisy Observationscited by 6
  2. 2025Recent Advances in Multi-Agent Human Trajectory Prediction: A Comprehensive Reviewcited by 2
  3. 2025Pedestrian Intention and Trajectory Prediction in Unstructured Traffic Using IDD-PeDcited by 1
  4. 2026ACME: A Multi-Cultural, Multi-Embodiment Social-Navigation Datasetcited by 0
  5. 2026Kairos: Grounded Forecasting of Presence and Directional Flow in 4D Scene Graphscited by 0
  6. 2026Probabilistic Pedestrian Forecasts from a Handheld Phone: World-Frame Heat Maps, Visual-Inertial Height Drift, and Evaluation without Ground Truthcited by 0