mlsanityData to 7 Oct 2026

Explainable Action Prediction through Self-Supervision on Scene Graphs

arXiv 2302.03477 ↗First posted 7 Feb 2023cs.CV cs.ROcited by 9

Not on the map. Why?

Abstract (arXiv)

This work explores scene graphs as a distilled representation of high-level information for autonomous driving, applied to future driver-action prediction. Given the scarcity and strong imbalance of data samples, we propose a self-supervision pipeline to infer representative and well-separated embeddings. Key aspects are interpretability and explainability; as such, we embed in our architecture attention mechanisms that can create spatial and temporal heatmaps on the scene graphs. We evaluate our system on the ROAD dataset against a fully-supervised approach, showing the superiority of our training regime.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 28 of its 33 references that are in the map. First posted on arXiv 7 Feb 2023.

Builds on

28 of 33 references.

References · 28

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  4. 2018How Powerful are Graph Neural Networks?cited by 4,355
  5. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  6. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  7. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  8. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  9. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  10. 2018Deep Graph Infomaxcited by 1,101
  11. 2020Deep Graph Contrastive Representation Learningcited by 535
  12. 2019InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximizationcited by 441
16 more
  1. 2019Self-Attention Graph Poolingcited by 421
  2. 2021Graph Self-Supervised Learning: A Surveycited by 289
  3. 2018Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoningcited by 102
  4. 2021Spatial-Temporal Transformer for Dynamic Scene Graph Generationcited by 95
  5. 2019Learning Actor Relation Graphs for Group Activity Recognitioncited by 83
  6. 2019Relational Graph Attention Networkscited by 81
  7. 2021Explanations in Autonomous Driving: A Surveycited by 80
  8. 2021ROAD: The ROad event Awareness Dataset for Autonomous Drivingcited by 54
  9. 2019Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Trafficcited by 50
  10. 2020Scene-Graph Augmented Data-Driven Risk Assessment of Autonomous Vehicle Decisionscited by 34
  11. 2020Structured Query-Based Image Retrieval Using Scene Graphscited by 33
  12. 2021Spatio-Temporal Scene-Graph Embedding for Autonomous Vehicle Collision Predictioncited by 29
  13. 2019Learning 3D-aware Egocentric Spatial-Temporal Interaction via Graph Convolutional Networkscited by 22
  14. 2021roadscene2vec: A Tool for Extracting and Embedding Road Scene-Graphscited by 15
  15. 2020Understanding Dynamic Scenes using Graph Convolution Networkscited by 10
  16. 2020Towards Accurate Vehicle Behaviour Classification With Multi-Relational Graph Convolutional Networkscited by 8

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 3; 2025: 3.

9 papers.

  1. 2024Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Reviewcited by 52
  2. 2023Action Scene Graphs for Long-Form Understanding of Egocentric Videoscited by 51
  3. 2023RS2G: Data-Driven Scene-Graph Extraction and Embedding for Robust Autonomous Perception and Scenario Understandingcited by 17
  4. 2024Advancing Explainable Autonomous Vehicle Systems: A Comprehensive Review and Research Roadmapcited by 6
  5. 2023Temporal DINO: A Self-supervised Video Strategy to Enhance Action Predictioncited by 3
  6. 2025Explainable Scene Understanding with Qualitative Representations and Graph Neural Networkscited by 1
  7. 2025Leveraging Foundation Models for Multimodal Graph-Based Action Recognitioncited by 1
  8. 2024GraphSCENE: On-Demand Critical Scenario Generation for Autonomous Vehicles in Simulationcited by 0
  9. 2025Hierarchical Feature-level Reverse Propagation for Post-Training Neural Networkscited by 0