# Dynamics-Aware Spatiotemporal Occupancy Prediction in Urban Environments

arXiv: https://arxiv.org/abs/2209.13172 · HTML: https://mlsanity.com/paper/2209.13172/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 21 references that are in the map.
- First posted on arXiv 27 Sep 2022.

## Builds on (most cited first, 14 of 14)

- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving](https://mlsanity.com/paper/2003.03653/) (2020)
- [Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning](https://mlsanity.com/paper/1605.08104/) (2016)
- [PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning](https://mlsanity.com/paper/1804.06300/) (2018)
- [Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spatiotemporal Dynamics](https://mlsanity.com/paper/1811.07490/) (2018)
- [Moving Object Segmentation in 3D LiDAR Data: A Learning-based Approach Exploiting Sequential Data](https://mlsanity.com/paper/2105.08971/) (2021)
- [Long-Term Occupancy Grid Prediction Using Recurrent Neural Networks](https://mlsanity.com/paper/1809.03782/) (2018)
- [Multi-Step Prediction of Occupancy Grid Maps with Recurrent Neural Networks](https://mlsanity.com/paper/1812.09395/) (2018)
- [Dynamic Environment Prediction in Urban Scenes using Recurrent Representation Learning](https://mlsanity.com/paper/1904.12374/) (2019)
- [A Random Finite Set Approach for Dynamic Occupancy Grid Maps with Real-Time Application](https://mlsanity.com/paper/1605.02406/) (2016)
- [Double-Prong ConvLSTM for Spatiotemporal Occupancy Prediction in Dynamic Environments](https://mlsanity.com/paper/2011.09045/) (2020)
- [Attention Augmented ConvLSTM for Environment Prediction](https://mlsanity.com/paper/2010.09662/) (2020)
- [Robust Moving Objects Detection in Lidar Data Exploiting Visual Cues](https://mlsanity.com/paper/1609.09267/) (2016)

## Built on by (most cited first, 17 of 17)

- [Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Driving](https://mlsanity.com/paper/2408.14197/) (2024)
- [A Survey of World Models for Autonomous Driving](https://mlsanity.com/paper/2501.11260/) (2025)
- [Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applications](https://mlsanity.com/paper/2311.17663/) (2023)
- [UniOcc: A Unified Benchmark for Occupancy Forecasting and Prediction in Autonomous Driving](https://mlsanity.com/paper/2503.24381/) (2025)
- [CMP: Cooperative Motion Prediction with Multi-Agent Communication](https://mlsanity.com/paper/2403.17916/) (2024)
- [Pedestrian Crossing Action Recognition and Trajectory Prediction with 3D Human Keypoints](https://mlsanity.com/paper/2306.01075/) (2023)
- [Targeted Adversarial Attacks against Neural Network Trajectory Predictors](https://mlsanity.com/paper/2212.04138/) (2022)
- [DUFOMap: Efficient Dynamic Awareness Mapping](https://mlsanity.com/paper/2403.01449/) (2024)
- [A Dynamic Points Removal Benchmark in Point Cloud Maps](https://mlsanity.com/paper/2307.07260/) (2023)
- [Adaptive Prediction Ensemble: Improving Out-of-Distribution Generalization of Motion Forecasting](https://mlsanity.com/paper/2407.09475/) (2024)
- [Self-supervised Multi-future Occupancy Forecasting for Autonomous Driving](https://mlsanity.com/paper/2407.21126/) (2024)
- [ForecastOcc: Vision-based Semantic Occupancy Forecasting](https://mlsanity.com/paper/2602.08006/) (2026)
- [LEGO-Motion: Learning-Enhanced Grids with Occupancy Instance Modeling for Class-Agnostic Motion Prediction](https://mlsanity.com/paper/2503.07367/) (2025)
- [LOPR: Latent Occupancy PRediction using Generative Models](https://mlsanity.com/paper/2210.01249/) (2022)
- [Vehicle Motion Forecasting using Prior Information and Semantic-assisted Occupancy Grid Maps](https://mlsanity.com/paper/2308.04303/) (2023)
- [Flow-guided Motion Prediction with Semantics and Dynamic Occupancy Grid Maps](https://mlsanity.com/paper/2407.15675/) (2024)
- [Integrating Specialized and Generic Agent Motion Prediction with Dynamic Occupancy Grid Maps](https://mlsanity.com/paper/2602.07938/) (2026)

## Abstract (arXiv)

Detection and segmentation of moving obstacles, along with prediction of the future occupancy states of the local environment, are essential for autonomous vehicles to proactively make safe and informed decisions. In this paper, we propose a framework that integrates the two capabilities together using deep neural network architectures. Our method first detects and segments moving objects in the scene, and uses this information to predict the spatiotemporal evolution of the environment around autonomous vehicles. To address the problem of direct integration of both static-dynamic object segmentation and environment prediction models, we propose using occupancy-based environment representations across the whole framework. Our method is validated on the real-world Waymo Open Dataset and demonstrates higher prediction accuracy than baseline methods.