mlsanityData to 7 Oct 2026

InteractionNet: Joint Planning and Prediction for Autonomous Driving with Transformers

arXiv 2309.03475 ↗First posted 7 Sep 2023cs.RO cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Planning and prediction are two important modules of autonomous driving and have experienced tremendous advancement recently. Nevertheless, most existing methods regard planning and prediction as independent and ignore the correlation between them, leading to the lack of consideration for interaction and dynamic changes of traffic scenarios. To address this challenge, we propose InteractionNet, which leverages transformer to share global contextual reasoning among all traffic participants to capture interaction and interconnect planning and prediction to achieve joint. Besides, InteractionNet deploys another transformer to help the model pay extra attention to the perceived region containing critical or unseen vehicles. InteractionNet outperforms other baselines in several benchmarks, especially in terms of safety, which benefits from the joint consideration of planning and forecasting. The code will be available at https://github.com/fujiawei0724/InteractionNet.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 38 of its 47 references that are in the map. First posted on arXiv 7 Sep 2023.

Builds on

38 of 47 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  5. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  6. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  7. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  8. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
  9. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  10. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  11. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
  12. 2019Learning by Cheatingcited by 361
26 more
  1. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  2. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 247
  3. 2020Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Netcited by 228
  4. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  5. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 191
  6. 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
  7. 2020Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representationscited by 150
  8. 2019End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordancescited by 147
  9. 2020PnPNet: End-to-End Perception and Prediction with Tracking in the Loopcited by 143
  10. 2022PersFormer: 3D Lane Detection via Perspective Transformer and the OpenLane Benchmarkcited by 140
  11. 2020Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?cited by 139
  12. 2022Learning from All Vehiclescited by 135
  13. 2019Multi-Agent Tensor Fusion for Contextual Trajectory Predictioncited by 131
  14. 2021Learning to drive from a world on railscited by 114
  15. 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
  16. 2022KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradientscited by 95
  17. 2022M2I: From Factored Marginal Trajectory Prediction to Interactive Predictioncited by 81
  18. 2021Multi-modal Motion Prediction with Transformer-based Neural Network for Autonomous Drivingcited by 73
  19. 2020Temporal-Channel Transformer for 3D Lidar-Based Video Object Detection in Autonomous Drivingcited by 55
  20. 2022CAT-Det: Contrastively Augmented Transformer for Multi-modal 3D Object Detectioncited by 32
  21. 2022Perceive, Interact, Predict: Learning Dynamic and Static Clues for End-to-End Motion Predictioncited by 28
  22. 2022Augmenting Reinforcement Learning with Transformer-based Scene Representation Learning for Decision-making of Autonomous Drivingcited by 24
  23. 2021EPSILON: An Efficient Planning System for Automated Vehicles in Highly Interactive Environmentscited by 22
  24. 2019Safe Trajectory Generation for Complex Urban Environments Using Spatio-temporal Semantic Corridorcited by 16
  25. 2020A Two-Stage Optimization-based Motion Planner for Safe Urban Drivingcited by 9
  26. 2022MSF3DDETR: Multi-Sensor Fusion 3D Detection Transformer for Autonomous Drivingcited by 3

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 2.

4 papers.

  1. 2025UncAD: Towards Safe End-to-end Autonomous Driving via Online Map Uncertaintycited by 14
  2. 2025FocalAD: Local Motion Planning for End-to-End Autonomous Drivingcited by 6
  3. 2024PP-TIL: Personalized Planning for Autonomous Driving with Instance-based Transfer Imitation Learningcited by 3
  4. 2024METDrive: Multi-modal End-to-end Autonomous Driving with Temporal Guidancecited by 2