DSDNet: Deep Structured self-Driving Network
Not on the map. Why?
Abstract (arXiv)
In this paper, we propose the Deep Structured self-Driving Network (DSDNet), which performs object detection, motion prediction, and motion planning with a single neural network. Towards this goal, we develop a deep structured energy based model which considers the interactions between actors and produces socially consistent multimodal future predictions. Furthermore, DSDNet explicitly exploits the predicted future distributions of actors to plan a safe maneuver by using a structured planning cost. Our sample-based formulation allows us to overcome the difficulty in probabilistic inference of continuous random variables. Experiments on a number of large-scale self driving datasets demonstrate that our model significantly outperforms the state-of-the-art.
Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Aug 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 9; 2022: 12; 2023: 18; 2024: 5; 2025: 15; 2026: 2.
61 papers.
Sort:
By area:
- 2022Planning-oriented Autonomous Drivingcited by 1,068
- 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
- 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
- 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
- 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
- 2021AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehiclescited by 132
- 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
- 2021LaneRCNN: Distributed Representations for Graph-Centric Motion Forecastingcited by 112
- 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 101
- 2021LookOut: Diverse Multi-Future Prediction and Planning for Self-Drivingcited by 96
- 2023ADAPT: Action-aware Driving Caption Transformercited by 78
- 2022GoRela: Go Relative for Viewpoint-Invariant Motion Forecastingcited by 68
38 more
- 2025A Survey of World Models for Autonomous Drivingcited by 60
- 2025STAMP: Scalable Task And Model-agnostic Collaborative Perceptioncited by 44
- 2024End-to-End Autonomous Driving through V2X Cooperationcited by 42
- 2022JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Drivingcited by 40
- 2023Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perceptioncited by 38
- 2021Divide-and-Conquer for Lane-Aware Diverse Trajectory Predictioncited by 37
- 2022DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehiclescited by 36
- 2023Rethinking Closed-loop Training for Autonomous Drivingcited by 29
- 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
- 2021Learning Interpretable End-to-End Vision-Based Motion Planning for Autonomous Driving with Optical Flow Distillationcited by 25
- 2025LangCoop: Collaborative Driving with Languagecited by 25
- 2021Deep Multi-Task Learning for Joint Localization, Perception, and Predictioncited by 22
- 2023MapPrior: Bird's-Eye View Map Layout Estimation with Generative Modelscited by 21
- 2021Deep Structured Reactive Planningcited by 18
- 2021End-to-End Interactive Prediction and Planning with Optical Flow Distillation for Autonomous Drivingcited by 17
- 2025CoCMT: Communication-Efficient Cross-Modal Transformer for Collaborative Perceptioncited by 13
- 2021Decoder Fusion RNN: Context and Interaction Aware Decoders for Trajectory Predictioncited by 12
- 2024QuAD: Query-based Interpretable Neural Motion Planning for Autonomous Drivingcited by 11
- 2025AirV2X: Unified Air-Ground Vehicle-to-Everything Collaborationcited by 11
- 2023Adv3D: Generating Safety-Critical 3D Objects through Closed-Loop Simulationcited by 10
- 2023Energy-based Potential Games for Joint Motion Forecasting and Controlcited by 9
- 2024V2X-PC: Vehicle-to-everything Collaborative Perception via Point Clustercited by 9
- 2022Trajectory Forecasting from Detection with Uncertainty-Aware Motion Encodingcited by 8
- 2023SCARP: 3D Shape Completion in ARbitrary Poses for Improved Graspingcited by 8
- 2025GCP: Guarded Collaborative Perception with Spatial-Temporal Aware Malicious Agent Detectioncited by 8
- 2025CoST: Efficient Collaborative Perception From Unified Spatiotemporal Perspectivecited by 7
- 2022Representing Spatial Trajectories as Distributionscited by 5
- 2025Automated Vehicles Should be Connected with Natural Languagecited by 5
- 2023Interpretable Motion Planner for Urban Driving via Hierarchical Imitation Learningcited by 4
- 2023On a Connection between Differential Games, Optimal Control, and Energy-based Models for Multi-Agent Interactionscited by 4
- 2025Interaction-Merged Motion Planning: Effectively Leveraging Diverse Motion Datasets for Robust Planningcited by 4
- 2025Perfect Prediction or Plenty of Proposals? What Matters Most in Planning for Autonomous Drivingcited by 4
- 2023DuEqNet: Dual-Equivariance Network in Outdoor 3D Object Detection for Autonomous Drivingcited by 3
- 2025Non-differentiable Reward Optimization for Diffusion-based Autonomous Motion Planningcited by 3
- 2026Learning Mutual View Information Graph for Adaptive Adversarial Collaborative Perceptioncited by 3
- 2022Fail-Safe Adversarial Generative Imitation Learningcited by 2
- 2022Enhancing the Transferability via Feature-Momentum Adversarial Attackcited by 2
- 20233D Surface Reconstruction in the Wild by Deforming Shape Priors from Synthetic Datacited by 2
50 most cited of 61 All 61 (JSON)
- 2026Unified Prediction and Planning via Conflict-Aware Disjoint Parameter Training22 Jul 2026
- 2026Learning Mutual View Information Graph for Adaptive Adversarial Collaborative Perception23 Feb 2026
- 2025SafeCoop: Unravelling Full Stack Safety in Agentic Collaborative Driving20 Oct 2025
- 2025Perfect Prediction or Plenty of Proposals? What Matters Most in Planning for Autonomous Driving17 Oct 2025
- 2025You Share Beliefs, I Adapt: Progressive Heterogeneous Collaborative Perception11 Sep 2025
- 2025CoST: Efficient Collaborative Perception From Unified Spatiotemporal Perspective1 Aug 2025
- 2025Non-differentiable Reward Optimization for Diffusion-based Autonomous Motion Planning17 Jul 2025
- 2025Interaction-Merged Motion Planning: Effectively Leveraging Diverse Motion Datasets for Robust Planning7 Jul 2025
- 2025Automated Vehicles Should be Connected with Natural Language29 Jun 2025
- 2025AirV2X: Unified Air-Ground Vehicle-to-Everything Collaboration24 Jun 2025
- 2025LangCoop: Collaborative Driving with Language18 Apr 2025
- 2025EP-Diffuser: An Efficient Diffusion Model for Traffic Scene Generation and Prediction via Polynomial Representations7 Apr 2025
38 more
- 2025CoCMT: Communication-Efficient Cross-Modal Transformer for Collaborative Perception13 Mar 2025
- 2025STAMP: Scalable Task And Model-agnostic Collaborative Perception24 Jan 2025
- 2025Knowledge-Informed Multi-Agent Trajectory Prediction at Signalized Intersections for Infrastructure-to-Everything23 Jan 2025
- 2025A Survey of World Models for Autonomous Driving20 Jan 2025
- 2025GCP: Guarded Collaborative Perception with Spatial-Temporal Aware Malicious Agent Detection5 Jan 2025
- 2024RuleFuser: An Evidential Bayes Approach for Rule Injection in Imitation Learned Planners and Predictors for Robustness under Distribution Shifts18 May 2024
- 2024GAD-Generative Learning for HD Map-Free Autonomous Driving1 May 2024
- 2024QuAD: Query-based Interpretable Neural Motion Planning for Autonomous Driving1 Apr 2024
- 2024End-to-End Autonomous Driving through V2X Cooperation31 Mar 2024
- 2024V2X-PC: Vehicle-to-everything Collaborative Perception via Point Cluster25 Mar 2024
- 2023Energy-based Potential Games for Joint Motion Forecasting and Control4 Dec 2023
- 2023Adv3D: Generating Safety-Critical 3D Objects through Closed-Loop Simulation2 Nov 2023
- 2023Pixel State Value Network for Combined Prediction and Planning in Interactive Environments11 Oct 2023
- 2023Semantic and Articulated Pedestrian Sensing Onboard a Moving Vehicle12 Sep 2023
- 2023Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perception31 Aug 2023
- 2023On a Connection between Differential Games, Optimal Control, and Energy-based Models for Multi-Agent Interactions31 Aug 2023
- 2023MapPrior: Bird's-Eye View Map Layout Estimation with Generative Models24 Aug 2023
- 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Review10 Aug 2023
- 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey10 Jul 2023
- 2023End-to-end Autonomous Driving: Challenges and Frontiers29 Jun 2023
- 2023Rethinking Closed-loop Training for Autonomous Driving27 Jun 2023
- 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusion5 Jun 2023
- 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving10 May 2023
- 2023Interpretable Motion Planner for Urban Driving via Hierarchical Imitation Learning24 Mar 2023
- 2023DuEqNet: Dual-Equivariance Network in Outdoor 3D Object Detection for Autonomous Driving27 Feb 2023
- 20233D Surface Reconstruction in the Wild by Deforming Shape Priors from Synthetic Data24 Feb 2023
- 2023ADAPT: Action-aware Driving Caption Transformer1 Feb 2023
- 2023SCARP: 3D Shape Completion in ARbitrary Poses for Improved Grasping17 Jan 2023
- 2022Planning-oriented Autonomous Driving20 Dec 2022
- 2022Flareon: Stealthy any2any Backdoor Injection via Poisoned Augmentation20 Dec 2022
- 2022JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Driving16 Dec 2022
- 2022DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehicles13 Dec 2022
- 2022GoRela: Go Relative for Viewpoint-Invariant Motion Forecasting4 Nov 2022
- 2022Representing Spatial Trajectories as Distributions4 Oct 2022
- 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning15 Jul 2022
- 20223D Object Detection for Autonomous Driving: A Comprehensive Survey19 Jun 2022
- 2022Why Did This Model Forecast This Future? Closed-Form Temporal Saliency Towards Causal Explanations of Probabilistic Forecasts1 Jun 2022
- 2022Enhancing the Transferability via Feature-Momentum Adversarial Attack22 Apr 2022
50 newest of 61 All 61 (JSON)
- 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
- 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
- 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
- 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 101
- 2021LookOut: Diverse Multi-Future Prediction and Planning for Self-Drivingcited by 96
- 2023ADAPT: Action-aware Driving Caption Transformercited by 78
- 2024End-to-End Autonomous Driving through V2X Cooperationcited by 42
- 2022JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Drivingcited by 40
- 2022DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehiclescited by 36
- 2023Rethinking Closed-loop Training for Autonomous Drivingcited by 29
- 2021Learning Interpretable End-to-End Vision-Based Motion Planning for Autonomous Driving with Optical Flow Distillationcited by 25
- 2025LangCoop: Collaborative Driving with Languagecited by 25
12 most cited of 17 All 17 (JSON)
- 2025SafeCoop: Unravelling Full Stack Safety in Agentic Collaborative Driving20 Oct 2025
- 2025Perfect Prediction or Plenty of Proposals? What Matters Most in Planning for Autonomous Driving17 Oct 2025
- 2025LangCoop: Collaborative Driving with Language18 Apr 2025
- 2024QuAD: Query-based Interpretable Neural Motion Planning for Autonomous Driving1 Apr 2024
- 2024End-to-End Autonomous Driving through V2X Cooperation31 Mar 2024
- 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey10 Jul 2023
- 2023End-to-end Autonomous Driving: Challenges and Frontiers29 Jun 2023
- 2023Rethinking Closed-loop Training for Autonomous Driving27 Jun 2023
- 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving10 May 2023
- 2023Interpretable Motion Planner for Urban Driving via Hierarchical Imitation Learning24 Mar 2023
- 2023ADAPT: Action-aware Driving Caption Transformer1 Feb 2023
- 2022JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Driving16 Dec 2022
12 newest of 17 All 17 (JSON)
- 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
- 2025A Survey of World Models for Autonomous Drivingcited by 60
- 2022Planning-oriented Autonomous Drivingcited by 1,068
- 2021AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehiclescited by 132
- 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
- 2021LaneRCNN: Distributed Representations for Graph-Centric Motion Forecastingcited by 112
- 2022GoRela: Go Relative for Viewpoint-Invariant Motion Forecastingcited by 68
- 2025STAMP: Scalable Task And Model-agnostic Collaborative Perceptioncited by 44
- 2023Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perceptioncited by 38
- 2021Divide-and-Conquer for Lane-Aware Diverse Trajectory Predictioncited by 37
- 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
- 2021Deep Multi-Task Learning for Joint Localization, Perception, and Predictioncited by 22
- 2023MapPrior: Bird's-Eye View Map Layout Estimation with Generative Modelscited by 21
- 2021Deep Structured Reactive Planningcited by 18
12 most cited of 38 All 38 (JSON)
- 2026Unified Prediction and Planning via Conflict-Aware Disjoint Parameter Training22 Jul 2026
- 2026Learning Mutual View Information Graph for Adaptive Adversarial Collaborative Perception23 Feb 2026
- 2025You Share Beliefs, I Adapt: Progressive Heterogeneous Collaborative Perception11 Sep 2025
- 2025CoST: Efficient Collaborative Perception From Unified Spatiotemporal Perspective1 Aug 2025
- 2025Interaction-Merged Motion Planning: Effectively Leveraging Diverse Motion Datasets for Robust Planning7 Jul 2025
- 2025Automated Vehicles Should be Connected with Natural Language29 Jun 2025
- 2025AirV2X: Unified Air-Ground Vehicle-to-Everything Collaboration24 Jun 2025
- 2025CoCMT: Communication-Efficient Cross-Modal Transformer for Collaborative Perception13 Mar 2025
- 2025STAMP: Scalable Task And Model-agnostic Collaborative Perception24 Jan 2025
- 2025Knowledge-Informed Multi-Agent Trajectory Prediction at Signalized Intersections for Infrastructure-to-Everything23 Jan 2025
- 2025GCP: Guarded Collaborative Perception with Spatial-Temporal Aware Malicious Agent Detection5 Jan 2025
- 2024GAD-Generative Learning for HD Map-Free Autonomous Driving1 May 2024
12 newest of 38 All 38 (JSON)