# Occupancy Prediction-Guided Neural Planner for Autonomous Driving arXiv: https://arxiv.org/abs/2305.03303 ยท HTML: https://mlsanity.com/paper/2305.03303/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Placed in Generative simulation for driving, one of 9 directions in End-to-end autonomous driving. - Builds on 31 of its 44 references that are in the map. - First posted on arXiv 5 May 2023. ## Placed in - End-to-end autonomous driving / [Generative simulation for driving](https://mlsanity.com/driving/generative-simulation-for-driving/) ## Builds on (most cited first, 30 of 31) - [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017) - [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021) - [ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst](https://mlsanity.com/paper/1812.03079/) (2018) - [DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets](https://mlsanity.com/paper/2108.09640/) (2021) - [MP3: A Unified Model to Map, Perceive, Predict and Plan](https://mlsanity.com/paper/2101.06806/) (2021) - [Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer](https://mlsanity.com/paper/2207.14024/) (2022) - [Scene Transformer: A unified architecture for predicting multiple agent trajectories](https://mlsanity.com/paper/2106.08417/) (2021) - [Self-Driving Cars: A Survey](https://mlsanity.com/paper/1901.04407/) (2019) - [GOHOME: Graph-Oriented Heatmap Output for future Motion Estimation](https://mlsanity.com/paper/2109.01827/) (2021) - [GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Driving](https://mlsanity.com/paper/2303.05760/) (2023) - [Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles](https://mlsanity.com/paper/2001.11231/) (2020) - [THOMAS: Trajectory Heatmap Output with learned Multi-Agent Sampling](https://mlsanity.com/paper/2110.06607/) (2021) - [LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving](https://mlsanity.com/paper/2101.06547/) (2021) - [M2I: From Factored Marginal Trajectory Prediction to Interactive Prediction](https://mlsanity.com/paper/2202.11884/) (2022) - [PiP: Planning-informed Trajectory Prediction for Autonomous Driving](https://mlsanity.com/paper/2003.11476/) (2020) - [Deep Imitative Models for Flexible Inference, Planning, and Control](https://mlsanity.com/paper/1810.06544/) (2018) - [Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveys](https://mlsanity.com/paper/2303.17220/) (2023) - [Differentiable Integrated Motion Prediction and Planning with Learnable Cost Function for Autonomous Driving](https://mlsanity.com/paper/2207.10422/) (2022) - [Occupancy Flow Fields for Motion Forecasting in Autonomous Driving](https://mlsanity.com/paper/2203.03875/) (2022) - [Driving Behavior Modeling using Naturalistic Human Driving Data with Inverse Reinforcement Learning](https://mlsanity.com/paper/2010.03118/) (2020) - [PredictionNet: Real-Time Joint Probabilistic Traffic Prediction for Planning, Control, and Simulation](https://mlsanity.com/paper/2109.11094/) (2021) - [Differentiable Gaussian Process Motion Planning](https://mlsanity.com/paper/1907.09591/) (2019) - [Deep Interactive Motion Prediction and Planning: Playing Games with Motion Prediction Models](https://mlsanity.com/paper/2204.02392/) (2022) - [Social Interactions for Autonomous Driving: A Review and Perspectives](https://mlsanity.com/paper/2208.07541/) (2022) - [Augmenting Reinforcement Learning with Transformer-based Scene Representation Learning for Decision-making of Autonomous Driving](https://mlsanity.com/paper/2208.12263/) (2022) - [An Integrated Framework of Decision Making and Motion Planning for Autonomous Vehicles Considering Social Behaviors](https://mlsanity.com/paper/2005.11059/) (2020) - [Interaction-Aware Trajectory Prediction of Connected Vehicles using CNN-LSTM Networks](https://mlsanity.com/paper/2005.12134/) (2020) - [StopNet: Scalable Trajectory and Occupancy Prediction for Urban Autonomous Driving](https://mlsanity.com/paper/2206.00991/) (2022) - [ReCoAt: A Deep Learning-based Framework for Multi-Modal Motion Prediction in Autonomous Driving Application](https://mlsanity.com/paper/2207.00726/) (2022) - [VectorFlow: Combining Images and Vectors for Traffic Occupancy and Flow Prediction](https://mlsanity.com/paper/2208.04530/) (2022) ## Built on by (most cited first, 8 of 8) - [Hybrid-Prediction Integrated Planning for Autonomous Driving](https://mlsanity.com/paper/2402.02426/) (2024) - [End-To-End Planning of Autonomous Driving in Industry and Academia: 2022-2023](https://mlsanity.com/paper/2401.08658/) (2023) - [An Overview about Emerging Technologies of Autonomous Driving](https://mlsanity.com/paper/2306.13302/) (2023) - [Spatially Visual Perception for End-to-End Robotic Learning](https://mlsanity.com/paper/2411.17458/) (2024) - [A Survey on Hybrid Motion Planning Methods for Automated Driving Systems](https://mlsanity.com/paper/2406.05575/) (2024) - [CAFE-AD: Cross-Scenario Adaptive Feature Enhancement for Trajectory Planning in Autonomous Driving](https://mlsanity.com/paper/2504.06584/) (2025) - [A Physics-informed End-to-End Occupancy Framework for Motion Planning of Autonomous Vehicles](https://mlsanity.com/paper/2505.07855/) (2025) - [CarPLAN: Context-Adaptive and Robust Planning with Dynamic Scene Awareness for Autonomous Driving](https://mlsanity.com/paper/2603.12607/) (2026) ## Abstract (arXiv) Forecasting the scalable future states of surrounding traffic participants in complex traffic scenarios is a critical capability for autonomous vehicles, as it enables safe and feasible decision-making. Recent successes in learning-based prediction and planning have introduced two primary challenges: generating accurate joint predictions for the environment and integrating prediction guidance for planning purposes. To address these challenges, we propose a two-stage integrated neural planning framework, termed OPGP, that incorporates joint prediction guidance from occupancy forecasting. The preliminary planning phase simultaneously outputs the predicted occupancy for various types of traffic actors based on imitation learning objectives, taking into account shared interactions, scene context, and actor dynamics within a unified Transformer structure. Subsequently, the transformed occupancy prediction guides optimization to further inform safe and smooth planning under Frenet coordinates. We train our planner using a large-scale, real-world driving dataset and validate it in open-loop configurations. Our proposed planner outperforms strong learning-based methods, exhibiting improved performance due to occupancy prediction guidance.