# Explainable Scene Understanding with Qualitative Representations and Graph Neural Networks

arXiv: https://arxiv.org/abs/2504.12817 · HTML: https://mlsanity.com/paper/2504.12817/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 42 references that are in the map.
- First posted on arXiv 17 Apr 2025.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [DriveLM: Driving with Graph Visual Question Answering](https://mlsanity.com/paper/2312.14150/) (2023)
- [Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructions](https://mlsanity.com/paper/2004.03967/) (2020)
- [A Comprehensive Survey of Scene Graphs: Generation and Application](https://mlsanity.com/paper/2104.01111/) (2021)
- [A Gentle Introduction to Deep Learning for Graphs](https://mlsanity.com/paper/1912.12693/) (2019)
- [Explainable and Explicit Visual Reasoning over Scene Graphs](https://mlsanity.com/paper/1812.01855/) (2018)
- [Graph Neural Networks for Modelling Traffic Participant Interaction](https://mlsanity.com/paper/1903.01254/) (2019)
- [Collaborative Dynamic 3D Scene Graphs for Automated Driving](https://mlsanity.com/paper/2309.06635/) (2023)
- [Explainable Video Action Reasoning via Prior Knowledge and State Transitions](https://mlsanity.com/paper/1908.10700/) (2019)
- [Visual Graphs from Motion (VGfM): Scene understanding with object geometry reasoning](https://mlsanity.com/paper/1807.05933/) (2018)
- [Lane Graph Estimation for Scene Understanding in Urban Driving](https://mlsanity.com/paper/2105.00195/) (2021)
- [SCENE: Reasoning about Traffic Scenes using Heterogeneous Graph Neural Networks](https://mlsanity.com/paper/2301.03512/) (2023)
- [Cooperative Behavior Planning for Automated Driving using Graph Neural Networks](https://mlsanity.com/paper/2202.11376/) (2022)
- [On Support Relations and Semantic Scene Graphs](https://mlsanity.com/paper/1609.05834/) (2016)
- [Explainable Action Prediction through Self-Supervision on Scene Graphs](https://mlsanity.com/paper/2302.03477/) (2023)
- [A Survey of Qualitative Spatial and Temporal Calculi -- Algebraic and Computational Properties](https://mlsanity.com/paper/1606.00133/) (2016)
- [Acquiring Qualitative Explainable Graphs for Automated Driving Scene Interpretation](https://mlsanity.com/paper/2308.12755/) (2023)
- [GSGFormer: Generative Social Graph Transformer for Multimodal Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2312.04479/) (2023)
- [Trustworthy Automated Driving through Qualitative Scene Understanding and Explanations](https://mlsanity.com/paper/2403.09668/) (2024)

## Built on by (most cited first, 1 of 1)

- [Attention over Scene Graphs: Indoor Scene Representations Toward CSAI Classification](https://mlsanity.com/paper/2509.26457/) (2025)

## Abstract (arXiv)

This paper investigates the integration of graph neural networks (GNNs) with Qualitative Explainable Graphs (QXGs) for scene understanding in automated driving. Scene understanding is the basis for any further reactive or proactive decision-making. Scene understanding and related reasoning is inherently an explanation task: why is another traffic participant doing something, what or who caused their actions? While previous work demonstrated QXGs' effectiveness using shallow machine learning models, these approaches were limited to analysing single relation chains between object pairs, disregarding the broader scene context. We propose a novel GNN architecture that processes entire graph structures to identify relevant objects in traffic scenes. We evaluate our method on the nuScenes dataset enriched with DriveLM's human-annotated relevance labels. Experimental results show that our GNN-based approach achieves superior performance compared to baseline methods. The model effectively handles the inherent class imbalance in relevant object identification tasks while considering the complete spatial-temporal relationships between all objects in the scene. Our work demonstrates the potential of combining qualitative representations with deep learning approaches for explainable scene understanding in autonomous driving systems.