mlsanityData to 7 Oct 2026

FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Cameras

arXiv 2104.10490 ↗First posted 21 Apr 2021cs.CV cs.ROcited by 239

Not on the map. Why?

Cited by 54 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Driving requires interacting with road agents and predicting their future behaviour in order to navigate safely. We present FIERY: a probabilistic future prediction model in bird's-eye view from monocular cameras. Our model predicts future instance segmentation and motion of dynamic agents that can be transformed into non-parametric future trajectories. Our approach combines the perception, sensor fusion and prediction components of a traditional autonomous driving stack by estimating bird's-eye-view prediction directly from surround RGB monocular camera inputs. FIERY learns to model the inherent stochastic nature of the future solely from camera driving data in an end-to-end manner, without relying on HD maps, and predicts multimodal future trajectories. We show that our model outperforms previous prediction baselines on the NuScenes and Lyft datasets. The code and trained models are available at https://github.com/wayveai/fiery.

Key facts. Cited by 239 papers in the map (data through 7 Oct 2026). Builds on 43 of its 53 references that are in the map. First posted on arXiv 21 Apr 2021.

Builds on

43 of 53 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2015Spatial Transformer Networkscited by 1,423
  5. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  6. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  7. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
  8. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 455
  9. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
  10. 2019Panoptic-DeepLabcited by 362
  11. 2017DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agentscited by 310
  12. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
31 more
  1. 2020Deep Continuous Fusion for Multi-Sensor 3D Object Detectioncited by 263
  2. 2019CoverNet: Multimodal Behavior Prediction using Trajectory Setscited by 259
  3. 2016A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehiclescited by 251
  4. 2017Stochastic Variational Video Predictioncited by 234
  5. 2020Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Netcited by 228
  6. 2015Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understandingcited by 218
  7. 2018Orthographic Feature Transform for Monocular 3D Object Detectioncited by 201
  8. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 178
  9. 2019Multiple Futures Predictioncited by 178
  10. 2020Predicting Semantic Map Representations from Images using Pyramid Occupancy Networkscited by 175
  11. 2019Cross-view Semantic Segmentation for Sensing Surroundingscited by 173
  12. 2017PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimationcited by 170
  13. 2021IntentNet: Learning to Predict Intention from Raw Sensor Datacited by 158
  14. 2015Delving Deeper into Convolutional Networks for Learning Video Representationscited by 146
  15. 2020Video Panoptic Segmentationcited by 141
  16. 2019Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactionscited by 117
  17. 2019Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Datacited by 104
  18. 2020MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Mapscited by 100
  19. 2016Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypothesescited by 96
  20. 2018Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Drivingcited by 90
  21. 2018Monocular Semantic Occupancy Grid Mapping with Convolutional Variational Encoder-Decoder Networkscited by 84
  22. 2020FISHING Net: Future Inference of Semantic Heatmaps In Gridscited by 55
  23. 2017Predicting Deeper into the Future of Semantic Segmentationcited by 51
  24. 2020Probabilistic Future Prediction for Video Scene Understandingcited by 45
  25. 2020BEV-Seg: Bird's Eye View Semantic Segmentation Using Geometry and Semantic Point Cloudcited by 38
  26. 2019Monocular Plan View Networks for Autonomous Drivingcited by 37
  27. 2020MultiXNet: Multiclass Multistage Multimodal Motion Predictioncited by 31
  28. 2018Generative Adversarial Frontal View to Bird View Synthesiscited by 25
  29. 2017Predicting Scene Parsing and Motion Dynamics in the Futurecited by 20
  30. 2020LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusioncited by 18
  31. 2019On the Benefit of Adversarial Training for Monocular Depth Estimationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 37; 2023: 68; 2024: 70; 2025: 38; 2026: 22.

239 papers.

Sort:

By area:

  1. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  2. 2022Planning-oriented Autonomous Drivingcited by 1,068
  3. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  4. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
  5. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 404
  6. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
  7. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  8. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  9. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
  10. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 310
  11. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  12. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 287
38 more
  1. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 287
  2. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Drivingcited by 230
  3. 2022CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformerscited by 227
  4. 2024GenAD: Generative End-to-End Autonomous Drivingcited by 225
  5. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  6. 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
  7. 2022Model-Based Imitation Learning for Urban Drivingcited by 149
  8. 2022PolarFormer: Multi-camera 3D Object Detection with Polar Transformercited by 147
  9. 2022Learning from All Vehiclescited by 135
  10. 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
  11. 2022Simple-BEV: What Really Matters for Multi-Sensor BEV Perception?cited by 128
  12. 2022Vision-Centric BEV Perception: A Surveycited by 126
  13. 2024VLP: Vision Language Planning for Autonomous Drivingcited by 125
  14. 2022BEVSegFormer: Bird's Eye View Semantic Segmentation From Arbitrary Camera Rigscited by 123
  15. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  16. 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 116
  17. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 114
  18. 2023Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusioncited by 105
  19. 2024textitS3Gaussian: Self-Supervised Street Gaussians for Autonomous Drivingcited by 105
  20. 2023Street-View Image Generation from a Bird's-Eye View Layoutcited by 102
  21. 2023BEVControl: Accurately Controlling Street-view Elements with Multi-perspective Consistency via BEV Sketch Layoutcited by 102
  22. 2022KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradientscited by 95
  23. 2024SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Predictioncited by 94
  24. 2023ReasonNet: End-to-End Driving with Temporal and Global Reasoningcited by 92
  25. 2023Symphonize 3D Semantic Scene Completion with Contextual Instance Queriescited by 88
  26. 2023Talk2BEV: Language-enhanced Bird's-eye View Maps for Autonomous Drivingcited by 88
  27. 2023CRN: Camera Radar Net for Accurate, Robust, Efficient 3D Perceptioncited by 81
  28. 2022Efficient and Robust 2D-to-BEV Representation Learning via Geometry-guided Kernel Transformercited by 80
  29. 2023End-to-End Vectorized HD-map Construction with Piecewise Bezier Curvecited by 80
  30. 2023ADAPT: Action-aware Driving Caption Transformercited by 78
  31. 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Drivingcited by 76
  32. 2023DrivingDiffusion: Layout-Guided multi-view driving scene video generation with latent diffusion modelcited by 75
  33. 2023DeepAccident: A Motion and Accident Prediction Benchmark for V2X Autonomous Drivingcited by 73
  34. 2023FusionAD: Multi-modality Fusion for Prediction and Planning Tasks of Autonomous Drivingcited by 67
  35. 2023Exploring Recurrent Long-term Temporal Fusion for Multi-view 3D Perceptioncited by 65
  36. 2025A Survey of World Models for Autonomous Drivingcited by 60
  37. 2023Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Trackingcited by 54
  38. 2023Online Map Vectorization for Autonomous Driving: A Rasterization Perspectivecited by 54

50 most cited of 239 All 239 (JSON)