# Predicting Future Occupancy Grids in Dynamic Environment with Spatio-Temporal Learning

arXiv: https://arxiv.org/abs/2205.03212 · HTML: https://mlsanity.com/paper/2205.03212/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 20 references that are in the map.
- First posted on arXiv 6 May 2022.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Unsupervised Learning for Physical Interaction through Video Prediction](https://mlsanity.com/paper/1605.07157/) (2016)
- [Stochastic Video Generation with a Learned Prior](https://mlsanity.com/paper/1802.07687/) (2018)
- [PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning](https://mlsanity.com/paper/2103.09504/) (2021)
- [A Review on Deep Learning Techniques for Video Prediction](https://mlsanity.com/paper/2004.05214/) (2020)
- [Dynamic Occupancy Grid Prediction for Urban Autonomous Driving: A Deep Learning Approach with Fully Automatic Labeling](https://mlsanity.com/paper/1705.08781/) (2017)
- [Long-Term Occupancy Grid Prediction Using Recurrent Neural Networks](https://mlsanity.com/paper/1809.03782/) (2018)
- [Multi-Step Prediction of Occupancy Grid Maps with Recurrent Neural Networks](https://mlsanity.com/paper/1812.09395/) (2018)
- [Dynamic Environment Prediction in Urban Scenes using Recurrent Representation Learning](https://mlsanity.com/paper/1904.12374/) (2019)
- [Deep Tracking: Seeing Beyond Seeing Using Recurrent Neural Networks](https://mlsanity.com/paper/1602.00991/) (2016)
- [Double-Prong ConvLSTM for Spatiotemporal Occupancy Prediction in Dynamic Environments](https://mlsanity.com/paper/2011.09045/) (2020)
- [Deep Tracking on the Move: Learning to Track the World from a Moving Vehicle using Recurrent Neural Networks](https://mlsanity.com/paper/1609.09365/) (2016)

## Built on by (most cited first, 7 of 7)

- [The Role of Predictive Uncertainty and Diversity in Embodied AI and Robot Learning](https://mlsanity.com/paper/2405.03164/) (2024)
- [Allo-centric Occupancy Grid Prediction for Urban Traffic Scene Using Video Prediction Networks](https://mlsanity.com/paper/2301.04454/) (2023)
- [Vehicle Motion Forecasting using Prior Information and Semantic-assisted Occupancy Grid Maps](https://mlsanity.com/paper/2308.04303/) (2023)
- [Flow-guided Motion Prediction with Semantics and Dynamic Occupancy Grid Maps](https://mlsanity.com/paper/2407.15675/) (2024)
- [STLight: a Fully Convolutional Approach for Efficient Predictive Learning by Spatio-Temporal joint Processing](https://mlsanity.com/paper/2411.10198/) (2024)
- [Integrating Specialized and Generic Agent Motion Prediction with Dynamic Occupancy Grid Maps](https://mlsanity.com/paper/2602.07938/) (2026)
- [ROSA-RL: Uncertainty-Aware Roundabout Optimized Speed Advisory with Reinforcement Learning](https://mlsanity.com/paper/2606.16558/) (2026)

## Abstract (arXiv)

Reliably predicting future occupancy of highly dynamic urban environments is an important precursor for safe autonomous navigation. Common challenges in the prediction include forecasting the relative position of other vehicles, modelling the dynamics of vehicles subjected to different traffic conditions, and vanishing surrounding objects. To tackle these challenges, we propose a spatio-temporal prediction network pipeline that takes the past information from the environment and semantic labels separately for generating future occupancy predictions. Compared to the current SOTA, our approach predicts occupancy for a longer horizon of 3 seconds and in a relatively complex environment from the nuScenes dataset. Our experimental results demonstrate the ability of spatio-temporal networks to understand scene dynamics without the need for HD-Maps and explicit modeling dynamic objects. We publicly release our occupancy grid dataset based on nuScenes to support further research.