# Long-Term Occupancy Grid Prediction Using Recurrent Neural Networks

arXiv: https://arxiv.org/abs/1809.03782 · HTML: https://mlsanity.com/paper/1809.03782/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Sep 2018.

## Built on by (most cited first, 29 of 29)

- [A Survey of World Models for Autonomous Driving](https://mlsanity.com/paper/2501.11260/) (2025)
- [Self-Supervised Pillar Motion Learning for Autonomous Driving](https://mlsanity.com/paper/2104.08683/) (2021)
- [Machine Learning for Autonomous Vehicle's Trajectory Prediction: A comprehensive survey, Challenges, and Future Research Directions](https://mlsanity.com/paper/2307.07527/) (2023)
- [LiDAR-based 4D Occupancy Completion and Forecasting](https://mlsanity.com/paper/2310.11239/) (2023)
- [Dynamics-Aware Spatiotemporal Occupancy Prediction in Urban Environments](https://mlsanity.com/paper/2209.13172/) (2022)
- [Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directions](https://mlsanity.com/paper/2503.03262/) (2025)
- [EgoNav: Egocentric Scene-aware Human Trajectory Prediction](https://mlsanity.com/paper/2403.19026/) (2024)
- [Predicting Future Occupancy Grids in Dynamic Environment with Spatio-Temporal Learning](https://mlsanity.com/paper/2205.03212/) (2022)
- [Self-supervised Multi-future Occupancy Forecasting for Autonomous Driving](https://mlsanity.com/paper/2407.21126/) (2024)
- [Sequential Point Clouds: A Survey](https://mlsanity.com/paper/2204.09337/) (2022)
- [Semi-Supervised Class-Agnostic Motion Prediction with Pseudo Label Regeneration and BEVMix](https://mlsanity.com/paper/2312.08009/) (2023)
- [Self-Supervised Bird's Eye View Motion Prediction with Cross-Modality Signals](https://mlsanity.com/paper/2401.11499/) (2024)
- [Self-Supervised Class-Agnostic Motion Prediction with Spatial and Temporal Consistency Regularizations](https://mlsanity.com/paper/2403.13261/) (2024)
- [LOPR: Latent Occupancy PRediction using Generative Models](https://mlsanity.com/paper/2210.01249/) (2022)
- [SLPC: a VRNN-based approach for stochastic lidar prediction and completion in autonomous driving](https://mlsanity.com/paper/2102.09883/) (2021)
- [Allo-centric Occupancy Grid Prediction for Urban Traffic Scene Using Video Prediction Networks](https://mlsanity.com/paper/2301.04454/) (2023)
- [Vehicle Motion Forecasting using Prior Information and Semantic-assisted Occupancy Grid Maps](https://mlsanity.com/paper/2308.04303/) (2023)
- [Vehicle trajectory prediction in top-view image sequences based on deep learning method](https://mlsanity.com/paper/2102.01749/) (2021)
- [Self-Supervised 3D Scene Flow Estimation and Motion Prediction using Local Rigidity Prior](https://mlsanity.com/paper/2310.11284/) (2023)
- [Are you a robot? Detecting Autonomous Vehicles from Behavior Analysis](https://mlsanity.com/paper/2403.09571/) (2024)
- [Flow-guided Motion Prediction with Semantics and Dynamic Occupancy Grid Maps](https://mlsanity.com/paper/2407.15675/) (2024)
- [How Cars Move: Analyzing Driving Dynamics for Safer Urban Traffic](https://mlsanity.com/paper/2412.04020/) (2024)
- [Weakly and Self-Supervised Class-Agnostic Motion Prediction for Autonomous Driving](https://mlsanity.com/paper/2509.13116/) (2025)
- [Deep Occupancy-Predictive Representations for Autonomous Driving](https://mlsanity.com/paper/2303.04218/) (2023)
- [Memory-enhanced Invariant Prompt Learning for Urban Flow Prediction under Distribution Shifts](https://mlsanity.com/paper/2412.05534/) (2024)
- [ObserveNet Control: A Vision-Dynamics Learning Approach to Predictive Control in Autonomous Vehicles](https://mlsanity.com/paper/2107.08690/) (2021)
- [Deep Learning Method for Cell-Wise Object Tracking, Velocity Estimation and Projection of Sensor Data over Time](https://mlsanity.com/paper/2306.06126/) (2023)
- [Integrating Specialized and Generic Agent Motion Prediction with Dynamic Occupancy Grid Maps](https://mlsanity.com/paper/2602.07938/) (2026)
- [Learning Humanoid Navigation from Human Data](https://mlsanity.com/paper/2604.00416/) (2026)

## Abstract (arXiv)

We tackle the long-term prediction of scene evolution in a complex downtown scenario for automated driving based on Lidar grid fusion and recurrent neural networks (RNNs). A bird's eye view of the scene, including occupancy and velocity, is fed as a sequence to a RNN which is trained to predict future occupancy. The nature of prediction allows generation of multiple hours of training data without the need of manual labeling. Thus, the training strategy and loss function is designed for long sequences of real-world data (unbalanced, continuously changing situations, false labels, etc.). The deep CNN architecture comprises convolutional long short-term memories (ConvLSTMs) to separate static from dynamic regions and to predict dynamic objects in future frames. Novel recurrent skip connections show the ability to predict small occluded objects, i.e. pedestrians, and occluded static regions. Spatio-temporal correlations between grid cells are exploited to predict multimodal future paths and interactions between objects. Experiments also quantify improvements to our previous network, a Monte Carlo approach, and literature.