mlsanityData to 7 Oct 2026

Learning Road Scene-level Representations via Semantic Region Prediction

arXiv 2301.00714 ↗First posted 2 Jan 2023cs.CV cs.ROcited by 4

Not on the map. Why?

Abstract (arXiv)

In this work, we tackle two vital tasks in automated driving systems, i.e., driver intent prediction and risk object identification from egocentric images. Mainly, we investigate the question: what would be good road scene-level representations for these two tasks? We contend that a scene-level representation must capture higher-level semantic and geometric representations of traffic scenes around ego-vehicle while performing actions to their destinations. To this end, we introduce the representation of semantic regions, which are areas where ego-vehicles visit while taking an afforded action (e.g., left-turn at 4-way intersections). We propose to learn scene-level representations via a novel semantic region prediction task and an automatic semantic region labeling algorithm. Extensive evaluations are conducted on the HDD and nuScenes datasets, and the learned representations lead to state-of-the-art performance for driver intention prediction and risk object identification.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 42 of its 56 references that are in the map. First posted on arXiv 2 Jan 2023.

Builds on

42 of 56 references.

References · 42

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2017Mask R-CNNcited by 7,259
  6. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  7. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  8. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
  9. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
  10. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  11. 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 908
  12. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
30 more
  1. 2018Image Inpainting for Irregular Holes Using Partial Convolutionscited by 505
  2. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  3. 2019Video Instance Segmentationcited by 391
  4. 2020Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentationcited by 326
  5. 2015Structural-RNN: Deep Learning on Spatio-Temporal Graphscited by 234
  6. 2020Predicting Semantic Map Representations from Images using Pyramid Occupancy Networkscited by 175
  7. 2021IntentNet: Learning to Predict Intention from Raw Sensor Datacited by 158
  8. 2020Semantically-Guided Representation Learning for Self-Supervised Monocular Depthcited by 134
  9. 2021Translating Images into Mapscited by 115
  10. 2017Interpretable Learning for Self-Driving Cars by Visualizing Causal Attentioncited by 106
  11. 2018Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoningcited by 102
  12. 2021OadTR: Online Action Detection with Transformerscited by 83
  13. 2018TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactionscited by 82
  14. 2017Predicting the Driver's Focus of Attention: the DR(eye)VE Projectcited by 79
  15. 2018Temporal Recurrent Networks for Online Action Detectioncited by 79
  16. 2021Fine-grained Semantics-aware Representation Enhancement for Self-supervised Monocular Depth Estimationcited by 55
  17. 2015Recurrent Neural Networks for Driver Activity Anticipation via Sensory-Fusion Architecturecited by 42
  18. 2018Egocentric Vision-based Future Vehicle Localization for Intelligent Driving Assistance Systemscited by 42
  19. 2018Deep Object-Centric Policies for Autonomous Drivingcited by 41
  20. 2018Learning to Look around Objects for Top-View Representations of Outdoor Scenescited by 37
  21. 2020Who Make Drivers Stop? Towards Driver-centric Risk Assessment: Risk Object Identification via Causal Inferencecited by 34
  22. 2018Probabilistic Prediction of Vehicle Semantic Intention and Motioncited by 31
  23. 2019Learning 3D-aware Egocentric Spatial-Temporal Interaction via Graph Convolutional Networkscited by 22
  24. 2018A Parametric Top-View Representation of Complex Road Scenescited by 20
  25. 2019Goal-oriented Object Importance Estimation in On-road Driving Videoscited by 19
  26. 2020Interaction Graphs for Object Importance Estimation in On-road Driving Videoscited by 15
  27. 2021roadscene2vec: A Tool for Extracting and Embedding Road Scene-Graphscited by 15
  28. 2022Important Object Identification with Semi-Supervised Learning for Autonomous Drivingcited by 13
  29. 2020Driver Intention Anticipation Based on In-Cabin and Driving Scene Monitoringcited by 12
  30. 2021Environment Predictive Coding for Embodied Agentscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 0; 2026: 2.

4 papers.

  1. 2023Action-slot: Visual Action-centric Representations for Multi-label Atomic Activity Recognition in Traffic Scenescited by 12
  2. 2024Potential Field as Scene Affordance for Behavior Change-Based Visual Risk Object Identificationcited by 2
  3. 2026Uncertainty-Aware Vision-based Risk Object Identification via Conformal Risk Tube Predictioncited by 0
  4. 2026Action-Slot: Structured Action-Centric Representation Learning for Multi-Agent Atomic Activity Understandingcited by 0