mlsanityData to 7 Oct 2026

StretchBEV: Stretching Future Instance Prediction Spatially and Temporally

arXiv 2203.13641 ↗First posted 25 Mar 2022cs.CV cs.LGcited by 47

Not on the map. Why?

Abstract (arXiv)

In self-driving, predicting future in terms of location and motion of all the agents around the vehicle is a crucial requirement for planning. Recently, a new joint formulation of perception and prediction has emerged by fusing rich sensory information perceived from multiple cameras into a compact bird's-eye view representation to perform prediction. However, the quality of future predictions degrades over time while extending to longer time horizons due to multiple plausible predictions. In this work, we address this inherent uncertainty in future predictions with a stochastic temporal model. Our model learns temporal dynamics in a latent space through stochastic residual updates at each time step. By sampling from a learned distribution at each time step, we obtain more diverse future predictions that are also more accurate compared to previous work, especially stretching both spatially further regions in the scene and temporally over longer time horizons. Despite separate processing of each time step, our model is still efficient through decoupling of the learning of dynamics and the generation of future predictions.

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). Builds on 33 of its 41 references that are in the map. First posted on arXiv 25 Mar 2022.

Builds on

33 of 41 references.

References · 33

  1. 2017Squeeze-and-Excitation Networkscited by 4,763
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2018Neural Ordinary Differential Equationscited by 3,733
  5. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  6. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
  7. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
  8. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
  9. 2020Learning Lane Graph Representations for Motion Forecastingcited by 380
  10. 2019Panoptic-DeepLabcited by 362
  11. 2020TNT: Target-driveN Trajectory Predictioncited by 345
  12. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 292
21 more
  1. 2018Stochastic Video Generation with a Learned Priorcited by 244
  2. 2017Stochastic Variational Video Predictioncited by 234
  3. 2020Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Netcited by 228
  4. 2016Structured Inference Networks for Nonlinear State Space Modelscited by 182
  5. 2019Multiple Futures Predictioncited by 178
  6. 2021IntentNet: Learning to Predict Intention from Raw Sensor Datacited by 158
  7. 2016Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Datacited by 154
  8. 2019Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactionscited by 117
  9. 2020Stochastic Latent Residual Video Predictioncited by 106
  10. 2019Improved Conditional VRNNs for Video Predictioncited by 104
  11. 2019Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Datacited by 104
  12. 2019Unsupervised Learning of Object Structure and Dynamics from Videoscited by 92
  13. 2019High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networkscited by 88
  14. 2020FISHING Net: Future Inference of Semantic Heatmaps In Gridscited by 55
  15. 2018Temporal Difference Variational Auto-Encodercited by 48
  16. 2020Probabilistic Future Prediction for Video Scene Understandingcited by 45
  17. 2019DiversityGAN: Diversity-Aware Vehicle Motion Prediction via Latent Semantic Samplingcited by 40
  18. 2021SLAMP: Stochastic Latent Appearance and Motion Predictioncited by 32
  19. 2020MultiXNet: Multiclass Multistage Multimodal Motion Predictioncited by 31
  20. 2020SMART: Simultaneous Multi-Agent Recurrent Trajectory Predictioncited by 15
  21. 2022Stochastic Video Prediction with Structure and Motioncited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 17; 2024: 10; 2025: 9; 2026: 4.

47 papers.

Sort:

By area:

  1. 2022Planning-oriented Autonomous Drivingcited by 1,068
  2. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 404
  3. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 310
  4. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 287
  5. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  6. 2022Vision-Centric BEV Perception: A Surveycited by 126
  7. 2024VLP: Vision Language Planning for Autonomous Drivingcited by 125
  8. 2023Street-View Image Generation from a Bird's-Eye View Layoutcited by 102
  9. 2023BEVControl: Accurately Controlling Street-view Elements with Multi-perspective Consistency via BEV Sketch Layoutcited by 102
  10. 2023Visual Point Cloud Forecasting enables Scalable Autonomous Drivingcited by 97
  11. 2023FusionAD: Multi-modality Fusion for Prediction and Planning Tasks of Autonomous Drivingcited by 67
  12. 2025A Survey of World Models for Autonomous Drivingcited by 60
35 more
  1. 2023Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Trackingcited by 54
  2. 2023Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applicationscited by 50
  3. 2023OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environmentscited by 39
  4. 2022Behavioral Intention Prediction in Driving Scenes: A Surveycited by 30
  5. 2023TBP-Former: Learning Temporal Bird's-Eye-View Pyramid for Joint Perception and Prediction in Vision-Centric Autonomous Drivingcited by 30
  6. 2023Grid-Centric Traffic Scenario Perception for Autonomous Driving: A Comprehensive Reviewcited by 25
  7. 2022Are We Ready for Vision-Centric Driving Streaming Perception? The ASAP Benchmarkcited by 24
  8. 2023PowerBEV: A Powerful Yet Lightweight Framework for Instance Prediction in Bird's-Eye Viewcited by 24
  9. 2023PointBeV: A Sparse Approach to BeV Predictionscited by 23
  10. 2025Reinforced Refinement with Self-Aware Expansion for End-to-End Autonomous Drivingcited by 19
  11. 2024Hybrid-Prediction Integrated Planning for Autonomous Drivingcited by 18
  12. 2024OccFeat: Self-supervised Occupancy Feature Prediction for Pretraining BEV Segmentation Networkscited by 17
  13. 2024InverseMatrixVT3D: An Efficient Projection Matrix-Based Approach for 3D Occupancy Predictioncited by 14
  14. 2024MOTO: Offline Pre-training to Online Fine-tuning for Model-based Robot Learningcited by 13
  15. 2025Slot-Guided Adaptation of Pre-trained Diffusion Models for Object-Centric Learning and Compositional Generationcited by 13
  16. 2025Closing the Train-Test Gap in World Models for Gradient-Based Planningcited by 10
  17. 2024Joint Perception and Prediction for Autonomous Driving: A Surveycited by 7
  18. 2024Spatiotemporal Decoupling for Efficient Vision-Based Occupancy Forecastingcited by 6
  19. 2026ForecastOcc: Vision-based Semantic Occupancy Forecastingcited by 6
  20. 2024DA-BEV: Unsupervised Domain Adaptation for Bird's Eye View Perceptioncited by 5
  21. 2024FipTR: A Simple yet Effective Transformer Framework for Future Instance Prediction in Autonomous Drivingcited by 4
  22. 2022Multi-Camera Calibration Free BEV Representation for 3D Object Detectioncited by 3
  23. 2023StreamingFlow: Streaming Occupancy Forecasting with Asynchronous Multi-modal Data Streams via Neural Ordinary Differential Equationcited by 3
  24. 2023One Training for Multiple Deployments: Polar-based Adaptive BEV Perception for Autonomous Drivingcited by 3
  25. 2024Progressive Query Refinement Framework for Bird's-Eye-View Semantic Segmentation from Surrounding Imagescited by 2
  26. 2025Collision Risk Estimation via Loss Prediction in End-to-End Autonomous Drivingcited by 1
  27. 2025To New Beginnings: A Survey of Unified Perception in Autonomous Vehicle Softwarecited by 1
  28. 2025KEPT: Knowledge-Enhanced Prediction of Trajectories from Consecutive Driving Frames with Vision-Language Modelscited by 1
  29. 2026BEVPredFormer: Spatio-temporal Attention for BEV Instance Prediction in Autonomous Drivingcited by 1
  30. 2022Stochastic Future Prediction in Real World Driving Scenarioscited by 0
  31. 2023Online Camera-to-ground Calibration for Autonomous Drivingcited by 0
  32. 2025Compositional Video Synthesis by Temporal Object-Centric Learningcited by 0
  33. 2025BEVTraj: Map-Free End-to-End Trajectory Prediction in Bird's-Eye View with Deformable Attention and Sparse Goal Proposalscited by 0
  34. 2026TGRIP: A Text-Guided Approach to Vehicle Instance Prediction in Autonomous Drivingcited by 0
  35. 2026InterOCF: Spatio-Temporal 2D-3D Interaction for Camera-Only 4D Occupancy Forecastingcited by 0