From Camera-Based Sensing to Reasoning: A Comprehensive Review Toward Proactive Vulnerable Road User Safety
Not on the map. Why?
Abstract (arXiv)
Ensuring the safety of vulnerable road users (VRUs), such as pedestrians and cyclists, remains a critical challenge, as conventional infrastructure-based measures are often insufficient in dynamic urban environments. Recent advances in learning-based visual sensing systems have created new opportunities for more adaptive and context-aware VRU protection. This paper presents a comprehensive review of camera-based methods for proactive VRU safety, with a focus on developments over the past five years. Unlike prior surveys that primarily emphasize detection, we organize the literature into three interconnected components: visual perception (detection and classification), motion modeling (tracking and trajectory prediction), and behavior understanding (intent recognition and reasoning). These components form a unified hierarchical pipeline that enables early risk anticipation and timely intervention. Furthermore, this survey systematically incorporates emerging AI paradigms, including Vision Transformers (ViTs), Large Language Models (LLMs), and diffusion models, highlighting their roles in representation learning, uncertainty modeling, and semantic reasoning. Finally, we identify four key challenges specific to VRU safety, including data scarcity, behavioral uncertainty, edge deployment efficiency, and real-world sensing constraints, and discuss corresponding research directions. This work provides a unified foundation for the development of reliable, scalable, and deployable VRU safety systems.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 86 of its 157 references that are in the map. First posted on arXiv 30 Sep 2025.
Builds on
86 of 157 references.
References · 86
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
- 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
- 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,018
- 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 908
- 2016Simple Online and Realtime Trackingcited by 862
- 2021ByteTrack: Multi-Object Tracking by Associating Every Detection Boxcited by 812
- 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
- 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 687
- 2023DETRs Beat YOLOs on Real-time Object Detectioncited by 526
38 more
- 2021TrackFormer: Multi-Object Tracking with Transformerscited by 515
- 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 498
- 2020Sparse R-CNN: End-to-End Object Detection with Learnable Proposalscited by 494
- 2020FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Trackingcited by 489
- 2020Tracking Objects as Pointscited by 483
- 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 479
- 2022MaxViT: Multi-Axis Vision Transformercited by 476
- 2024YOLOv11: An Overview of the Key Architectural Enhancementscited by 414
- 2021MOTR: End-to-End Multiple-Object Tracking with Transformercited by 379
- 2020TransTrack: Multiple Object Tracking with Transformercited by 370
- 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
- 2022Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Trackingcited by 345
- 2020Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Predictioncited by 324
- 2022BoT-SORT: Robust Associations Multi-Pedestrian Trackingcited by 313
- 2019Towards Real-Time Multi-Object Trackingcited by 311
- 2025YOLOv12: Attention-Centric Real-Time Object Detectorscited by 300
- 2018CrowdHuman: A Benchmark for Detecting Human in a Crowdcited by 283
- 2021LLVIP: A Visible-infrared Paired Dataset for Low-light Visioncited by 229
- 2020Quasi-Dense Similarity Learning for Multiple Object Trackingcited by 220
- 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
- 2022Stochastic Trajectory Prediction via Motion Indeterminacy Diffusioncited by 184
- 2017CityPersons: A Diverse Dataset for Pedestrian Detectioncited by 171
- 2021Scene Transformer: A unified architecture for predicting multiple agent trajectoriescited by 165
- 2023Leapfrog Diffusion Model for Stochastic Trajectory Predictioncited by 129
- 2022GroupNet: Multiscale Hypergraph Neural Networks for Trajectory Prediction with Relational Reasoningcited by 128
- 2020Rethinking the competition between detection and ReID in Multi-Object Trackingcited by 101
- 2021Cross-Modality Fusion Transformer for Multispectral Object Detectioncited by 86
- 2020Multispectral Fusion for Object Detection with Cyclic Fuse-and-Refine Blockscited by 85
- 2020Improving Multispectral Pedestrian Detection by Addressing Modality Imbalance Problemscited by 82
- 2022PP-YOLOE: An evolved version of YOLOcited by 75
- 2018Illumination-aware Faster R-CNN for Robust Multispectral Pedestrian Detectioncited by 74
- 2024Selective Hourglass Mapping for Universal Image Restoration Based on Diffusion Modelcited by 73
- 2020Spatiotemporal Relationship Reasoning for Pedestrian Intent Predictioncited by 71
- 2023FasterViT: Fast Vision Transformers with Hierarchical Attentioncited by 65
- 2022QDTrack: Quasi-Dense Similarity Learning for Appearance-Only Multiple Object Trackingcited by 61
- 2020Detection in Crowded Scenes: One Proposal, Multiple Predictionscited by 59
- 2021Predicting Pedestrian Crossing Intention with Feature Fusion and Spatio-Temporal Attentioncited by 48
- 2023Scenario Diffusion: Controllable Driving Scenario Generation With Diffusioncited by 46
50 most cited of 86 All 86 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2019Predicting Motion of Vulnerable Road Users using High-Definition Maps and Efficient ConvNets
- 2025DRAMA-X: A Fine-grained Intent Prediction and Risk Reasoning Benchmark For Driving
- 2025INSIGHT: Enhancing Autonomous Driving Safety through Vision-Language Models on Context-Aware Hazard Detection and Edge Case Evaluation
- 2026Vision and Language: Novel Representations and Artificial intelligence for Driving Scene Safety Assessment and Autonomous Vehicle Planning
- 2026Urban Risk-Aware Navigation via VQA-Based Event Maps for People with Low Vision
- 2025VRU-Accident: A Vision-Language Benchmark for Video Question Answering and Dense Captioning for Accident Scene Understanding
- 2025Multi-view Phase-aware Pedestrian-Vehicle Incident Reasoning Framework with Vision-Language Models
- 2025HazardNet: A Small-Scale Vision Language Model for Real-Time Traffic Safety Detection at Edge Devices