# StopNet: Scalable Trajectory and Occupancy Prediction for Urban Autonomous Driving

arXiv: https://arxiv.org/abs/2206.00991 · HTML: https://mlsanity.com/paper/2206.00991/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 34 references that are in the map.
- First posted on arXiv 2 Jun 2022.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021)
- [VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation](https://mlsanity.com/paper/2005.04259/) (2020)
- [MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction](https://mlsanity.com/paper/1910.05449/) (2019)
- [ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst](https://mlsanity.com/paper/1812.03079/) (2018)
- [TNT: Target-driveN Trajectory Prediction](https://mlsanity.com/paper/2008.08294/) (2020)
- [DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents](https://mlsanity.com/paper/1704.04394/) (2017)
- [CoverNet: Multimodal Behavior Prediction using Trajectory Sets](https://mlsanity.com/paper/1911.10298/) (2019)
- [One Thousand and One Hours: Self-driving Motion Prediction Dataset](https://mlsanity.com/paper/2006.14480/) (2020)
- [Human Motion Trajectory Prediction: A Survey](https://mlsanity.com/paper/1905.06113/) (2019)
- [INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps](https://mlsanity.com/paper/1910.03088/) (2019)
- [MP3: A Unified Model to Map, Perceive, Predict and Plan](https://mlsanity.com/paper/2101.06806/) (2021)
- [Multiple Futures Prediction](https://mlsanity.com/paper/1911.00997/) (2019)
- [IntentNet: Learning to Predict Intention from Raw Sensor Data](https://mlsanity.com/paper/2101.07907/) (2021)
- [Multi-Agent Tensor Fusion for Contextual Trajectory Prediction](https://mlsanity.com/paper/1904.04776/) (2019)
- [Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactions](https://mlsanity.com/paper/1906.08945/) (2019)
- [Multi-Head Attention for Multi-Modal Joint Vehicle Motion Forecasting](https://mlsanity.com/paper/1910.03650/) (2019)
- [What-If Motion Prediction for Autonomous Driving](https://mlsanity.com/paper/2008.10587/) (2020)
- [Identifying Driver Interactions via Conditional Behavior Prediction](https://mlsanity.com/paper/2104.09959/) (2021)
- [Discrete Residual Flow for Probabilistic Pedestrian Behavior Prediction](https://mlsanity.com/paper/1910.08041/) (2019)
- [MultiXNet: Multiclass Multistage Multimodal Motion Prediction](https://mlsanity.com/paper/2006.02000/) (2020)
- [PRANK: motion Prediction based on RANKing](https://mlsanity.com/paper/2010.12007/) (2020)
- [PLOP: Probabilistic poLynomial Objects trajectory Planning for autonomous driving](https://mlsanity.com/paper/2003.08744/) (2020)
- [SMART: Simultaneous Multi-Agent Recurrent Trajectory Prediction](https://mlsanity.com/paper/2007.13078/) (2020)

## Built on by (most cited first, 14 of 14)

- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [Implicit Occupancy Flow Fields for Perception and Prediction in Self-Driving](https://mlsanity.com/paper/2308.01471/) (2023)
- [The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Review](https://mlsanity.com/paper/2308.05731/) (2023)
- [PointBeV: A Sparse Approach to BeV Predictions](https://mlsanity.com/paper/2312.00703/) (2023)
- [Hybrid-Prediction Integrated Planning for Autonomous Driving](https://mlsanity.com/paper/2402.02426/) (2024)
- [Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directions](https://mlsanity.com/paper/2503.03262/) (2025)
- [Adv3D: Generating Safety-Critical 3D Objects through Closed-Loop Simulation](https://mlsanity.com/paper/2311.01446/) (2023)
- [UniGen: Unified Modeling of Initial Agent States and Trajectories for Generating Autonomous Driving Scenarios](https://mlsanity.com/paper/2405.03807/) (2024)
- [STrajNet: Multi-modal Hierarchical Transformer for Occupancy Flow Field Prediction in Autonomous Driving](https://mlsanity.com/paper/2208.00394/) (2022)
- [Occupancy Prediction-Guided Neural Planner for Autonomous Driving](https://mlsanity.com/paper/2305.03303/) (2023)
- [VectorFlow: Combining Images and Vectors for Traffic Occupancy and Flow Prediction](https://mlsanity.com/paper/2208.04530/) (2022)
- [FutureNet-LOF: Joint Trajectory Prediction and Lane Occupancy Field Prediction with Future Context Encoding](https://mlsanity.com/paper/2406.14422/) (2024)
- [Pioneering SE(2)-Equivariant Trajectory Planning for Automated Driving](https://mlsanity.com/paper/2403.11304/) (2024)
- [The WayHome: Long-term Motion Prediction on Dynamically Scaled](https://mlsanity.com/paper/2310.04232/) (2023)

## Abstract (arXiv)

We introduce a motion forecasting (behavior prediction) method that meets the latency requirements for autonomous driving in dense urban environments without sacrificing accuracy. A whole-scene sparse input representation allows StopNet to scale to predicting trajectories for hundreds of road agents with reliable latency. In addition to predicting trajectories, our scene encoder lends itself to predicting whole-scene probabilistic occupancy grids, a complementary output representation suitable for busy urban environments. Occupancy grids allow the AV to reason collectively about the behavior of groups of agents without processing their individual trajectories. We demonstrate the effectiveness of our sparse input representation and our model in terms of computation and accuracy over three datasets. We further show that co-training consistent trajectory and occupancy predictions improves upon state-of-the-art performance under standard metrics.