# Attention Augmented ConvLSTM for Environment Prediction

arXiv: https://arxiv.org/abs/2010.09662 · HTML: https://mlsanity.com/paper/2010.09662/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2020.

## Built on by (most cited first, 12 of 12)

- [Interpretable Self-Aware Neural Networks for Robust Trajectory Prediction](https://mlsanity.com/paper/2211.08701/) (2022)
- [How Do We Fail? Stress Testing Perception in Autonomous Vehicles](https://mlsanity.com/paper/2203.14155/) (2022)
- [Dynamics-Aware Spatiotemporal Occupancy Prediction in Urban Environments](https://mlsanity.com/paper/2209.13172/) (2022)
- [Occlusion-Aware Crowd Navigation Using People as Sensors](https://mlsanity.com/paper/2210.00552/) (2022)
- [Self-supervised Multi-future Occupancy Forecasting for Autonomous Driving](https://mlsanity.com/paper/2407.21126/) (2024)
- [LOPR: Latent Occupancy PRediction using Generative Models](https://mlsanity.com/paper/2210.01249/) (2022)
- [Allo-centric Occupancy Grid Prediction for Urban Traffic Scene Using Video Prediction Networks](https://mlsanity.com/paper/2301.04454/) (2023)
- [Vehicle Motion Forecasting using Prior Information and Semantic-assisted Occupancy Grid Maps](https://mlsanity.com/paper/2308.04303/) (2023)
- [Flow-guided Motion Prediction with Semantics and Dynamic Occupancy Grid Maps](https://mlsanity.com/paper/2407.15675/) (2024)
- [RainDiff: End-to-end Precipitation Nowcasting Via Token-wise Attention Diffusion](https://mlsanity.com/paper/2510.14962/) (2025)
- [Deformation Driven Seq2Seq Longitudinal Tumor and Organs-at-Risk Prediction for Radiotherapy](https://mlsanity.com/paper/2106.09076/) (2021)
- [Integrating Specialized and Generic Agent Motion Prediction with Dynamic Occupancy Grid Maps](https://mlsanity.com/paper/2602.07938/) (2026)

## Abstract (arXiv)

Safe and proactive planning in robotic systems generally requires accurate predictions of the environment. Prior work on environment prediction applied video frame prediction techniques to bird's-eye view environment representations, such as occupancy grids. ConvLSTM-based frameworks used previously often result in significant blurring and vanishing of moving objects, thus hindering their applicability for use in safety-critical applications. In this work, we propose two extensions to the ConvLSTM to address these issues. We present the Temporal Attention Augmented ConvLSTM (TAAConvLSTM) and Self-Attention Augmented ConvLSTM (SAAConvLSTM) frameworks for spatiotemporal occupancy prediction, and demonstrate improved performance over baseline architectures on the real-world KITTI and Waymo datasets.