# GraphSCENE: On-Demand Critical Scenario Generation for Autonomous Vehicles in Simulation

arXiv: https://arxiv.org/abs/2410.13514 · HTML: https://mlsanity.com/paper/2410.13514/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 31 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting](https://mlsanity.com/paper/1709.04875/) (2017)
- [Heterogeneous Graph Attention Network](https://mlsanity.com/paper/1903.07293/) (2019)
- [Ragas: Automated Evaluation of Retrieval Augmented Generation](https://mlsanity.com/paper/2309.15217/) (2023)
- [GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models](https://mlsanity.com/paper/1802.08773/) (2018)
- [Efficient Graph Generation with Graph Recurrent Attention Networks](https://mlsanity.com/paper/1910.00760/) (2019)
- [Learning Deep Generative Models of Graphs](https://mlsanity.com/paper/1803.03324/) (2018)
- [TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarios](https://mlsanity.com/paper/2210.06609/) (2022)
- [ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehicles](https://mlsanity.com/paper/2405.14062/) (2024)
- [Language Conditioned Traffic Generation](https://mlsanity.com/paper/2307.07947/) (2023)
- [ROAD: The ROad event Awareness Dataset for Autonomous Driving](https://mlsanity.com/paper/2102.11585/) (2021)
- [Corner Case Generation and Analysis for Safety Assessment of Autonomous Vehicles](https://mlsanity.com/paper/2102.03483/) (2021)
- [Unconditional Scene Graph Generation](https://mlsanity.com/paper/2108.05884/) (2021)
- [Long-Tail Prediction Uncertainty Aware Trajectory Planning for Self-driving Vehicles](https://mlsanity.com/paper/2207.00788/) (2022)
- [One Ontology to Rule Them All: Corner Case Scenarios for Autonomous Driving](https://mlsanity.com/paper/2209.00342/) (2022)
- [Using Ontologies for the Formalization and Recognition of Criticality for Automated Driving](https://mlsanity.com/paper/2205.01532/) (2022)
- [Explainable Action Prediction through Self-Supervision on Scene Graphs](https://mlsanity.com/paper/2302.03477/) (2023)
- [1001 Ways of Scenario Generation for Testing of Self-driving Cars: A Survey](https://mlsanity.com/paper/2304.10850/) (2023)
- [Spatio-Temporal Latent Graph Structure Learning for Traffic Forecasting](https://mlsanity.com/paper/2202.12586/) (2022)
- [CC-SGG: Corner Case Scenario Generation using Learned Scene Graphs](https://mlsanity.com/paper/2309.09844/) (2023)

## Abstract (arXiv)

Testing and validating Autonomous Vehicle (AV) performance in safety-critical and diverse scenarios is crucial before real-world deployment. However, manually creating such scenarios in simulation remains a significant and time-consuming challenge. This work introduces a novel method that generates dynamic temporal scene graphs corresponding to diverse traffic scenarios, on-demand, tailored to user-defined preferences, such as AV actions, sets of dynamic agents, and criticality levels. A temporal Graph Neural Network (GNN) model learns to predict relationships between ego-vehicle, agents, and static structures, guided by real-world spatiotemporal interaction patterns and constrained by an ontology that restricts predictions to semantically valid links. Our model consistently outperforms the baselines in accurately generating links corresponding to the requested scenarios. We render the predicted scenarios in simulation to further demonstrate their effectiveness as testing environments for AV agents.