mlsanityData to 7 Oct 2026

GraphAD: Interaction Scene Graph for End-to-end Autonomous Driving

arXiv 2403.19098 ↗First posted 28 Mar 2024cs.CVcited by 25

On the map: End-to-end autonomous driving → Generative simulation for driving

Abstract (arXiv)

Modeling complicated interactions among the ego-vehicle, road agents, and map elements has been a crucial part for safety-critical autonomous driving. Previous works on end-to-end autonomous driving rely on the attention mechanism for handling heterogeneous interactions, which fails to capture the geometric priors and is also computationally intensive. In this paper, we propose the Interaction Scene Graph (ISG) as a unified method to model the interactions among the ego-vehicle, road agents, and map elements. With the representation of the ISG, the driving agents aggregate essential information from the most influential elements, including the road agents with potential collisions and the map elements to follow. Since a mass of unnecessary interactions are omitted, the more efficient scene-graph-based framework is able to focus on indispensable connections and leads to better performance. We evaluate the proposed method for end-to-end autonomous driving on the nuScenes dataset. Compared with strong baselines, our method significantly outperforms in the full-stack driving tasks, including perception, prediction, and planning. Code will be released at https://github.com/zhangyp15/GraphAD.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Placed in Generative simulation for driving, one of 9 directions in End-to-end autonomous driving. Builds on 39 of its 48 references that are in the map. First posted on arXiv 28 Mar 2024.

Builds on

39 of 48 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Graph Attention Networkscited by 8,854
  4. 2017Inductive Representation Learning on Large Graphscited by 6,284
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  6. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  7. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,199
  8. 2022Planning-oriented Autonomous Drivingcited by 1,068
  9. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  10. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  11. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
  12. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
27 more
  1. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  2. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  3. 2020Learning Lane Graph Representations for Motion Forecastingcited by 380
  4. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
  5. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
  6. 2023GPT-Driver: Learning to Drive with GPTcited by 344
  7. 2020Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloudcited by 335
  8. 2022MapTR: Structured Modeling and Learning for Online Vectorized HD Map Constructioncited by 311
  9. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 247
  10. 2021End-to-end Interpretable Neural Motion Plannercited by 240
  11. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  12. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  13. 2023Scene as Occupancycited by 208
  14. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 191
  15. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
  16. 2023A Language Agent for Autonomous Drivingcited by 161
  17. 2020PnPNet: End-to-End Perception and Prediction with Tracking in the Loopcited by 143
  18. 2022Learning from All Vehiclescited by 135
  19. 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 116
  20. 2020GNN3DMOT: Graph Neural Network for 3D Multi-Object Tracking with Multi-Feature Learningcited by 113
  21. 2022HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encodingcited by 109
  22. 2022MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queriescited by 90
  23. 2022Differentiable Raycasting for Self-supervised Occupancy Forecastingcited by 83
  24. 2021Monocular Quasi-Dense 3D Object Trackingcited by 80
  25. 2023FusionAD: Multi-modality Fusion for Prediction and Planning Tasks of Autonomous Drivingcited by 67
  26. 2021Object DGCNN: 3D Object Detection using Dynamic Graphscited by 58
  27. 2023Cross-Stream Contrastive Learning for Self-Supervised Skeleton-Based Action Recognitioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 13; 2026: 7.

25 papers.

Sort:

By area:

  1. 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 239
  2. 2024VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervisioncited by 85
  3. 2025Don't Shake the Wheel: Momentum-Aware Planning in End-to-End Autonomous Drivingcited by 84
  4. 2025A Survey on Vision-Language-Action Models for Autonomous Drivingcited by 72
  5. 2024Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Drivingcited by 67
  6. 2025iPad: Iterative Proposal-centric End-to-End Autonomous Drivingcited by 41
  7. 2025Universal Scene Graph Generationcited by 11
  8. 2026DriveMamba: Task-Centric Scalable State Space Model for Efficient End-to-End Autonomous Drivingcited by 9
  9. 2025MomaGraph: State-Aware Unified Scene Graphs with Vision-Language Model for Embodied Task Planningcited by 8
  10. 2025FocalAD: Local Motion Planning for End-to-End Autonomous Drivingcited by 6
  11. 2025GraphPilot: Grounded Scene Graph Conditioning for Language-Based Autonomous Drivingcited by 6
  12. 2025VR-Drive: Viewpoint-Robust End-to-End Driving with Feed-Forward 3D Gaussian Splattingcited by 5
13 more
  1. 2024LiDAR-based End-to-end Temporal Perception for Vehicle-Infrastructure Cooperationcited by 4
  2. 2024EgoFSD: Ego-Centric Fully Sparse Paradigm with Uncertainty Denoising and Iterative Refinement for Efficient End-to-End Self-Drivingcited by 3
  3. 2025Robo-SGG: Exploiting Layout-Oriented Normalization and Restitution Can Improve Robust Scene Graph Generationcited by 3
  4. 2025Echo Planning for Autonomous Driving: From Current Observations to Future Trajectories and Backcited by 3
  5. 2025Vision-Centric 4D Occupancy Forecasting and Planning via Implicit Residual World Modelscited by 3
  6. 2026Bridging Structure and Language: Graph-Based Visual Reasoning for Autonomous Road Understandingcited by 2
  7. 2025SUPER-AD: Semantic Uncertainty-aware Planning for End-to-End Robust Autonomous Drivingcited by 1
  8. 20252nd Place Solution for CVPR2024 E2E Challenge: End-to-End Autonomous Driving Using Vision Language Modelcited by 0
  9. 2026CURVE: Learning Causality-Inspired Invariant Representations for Robust Scene Understanding via Uncertainty-Guided Regularizationcited by 0
  10. 2026MOSA: Motion-Guided Semantic Alignment for Dynamic Scene Graph Generationcited by 0
  11. 2026NoPA: Non-Parametric Online 3D Scene Graph Generationcited by 0
  12. 2026Geometric 2D Scene Graph Generationcited by 0
  13. 2026VIPS: Vehicle-Infrastructure Cooperative Planning Benchmark via Pseudo-Simulationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023PPAD: Iterative Interactions of Prediction and Planning for End-to-end Autonomous Driving
  2. 2026GraphWorld: Long-Horizon Planning with World Models for End-to-End Autonomous Driving
  3. 2026FlowAD: Ego-Scene Interactive Modeling for Autonomous Driving
  4. 2024GenAD: Generative End-to-End Autonomous Driving
  5. 2026Causality-Aware End-to-End Autonomous Driving via Ego-Centric Joint Scene Modeling
  6. 2025Optimization-Guided Diffusion for Interactive Scene Generation
  7. 2026An interactive enhanced driving dataset for autonomous driving
  8. 2021Scene Transformer: A unified architecture for predicting multiple agent trajectories