mlsanityData to 14 Oct 2026

MVFuseNet: Improving End-to-End Object Detection and Motion Forecasting through Multi-View Fusion of LiDAR Data

arXiv 2104.10772 ↗First posted 21 Apr 2021cs.CV cs.ROcited by 18

Not on the map. Why?

Abstract (arXiv)

In this work, we propose textitMVFuseNet, a novel end-to-end method for joint object detection and motion forecasting from a temporal sequence of LiDAR data. Most existing methods operate in a single view by projecting data in either range view (RV) or bird's eye view (BEV). In contrast, we propose a method that effectively utilizes both RV and BEV for spatio-temporal feature learning as part of a temporal fusion network as well as for multi-scale feature learning in the backbone network. Further, we propose a novel sequential fusion approach that effectively utilizes multiple views in the temporal fusion network. We show the benefits of our multi-view approach for the tasks of detection and motion forecasting on two large-scale self-driving data sets, achieving state-of-the-art results. Furthermore, we show that MVFusenet scales well to large operating ranges while maintaining real-time performance.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 41 of its 48 references that are in the map. First posted on arXiv 21 Apr 2021.

Builds on

41 of 48 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  4. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  5. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  6. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  7. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 941
  8. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 724
  9. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 690
  10. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 530
  11. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 420
  12. 2020Learning Lane Graph Representations for Motion Forecastingcited by 382
29 more
  1. 2019PIXOR: Real-time 3D Object Detection from Point Cloudscited by 358
  2. 2020TNT: Target-driveN Trajectory Predictioncited by 347
  3. 2017DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agentscited by 310
  4. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
  5. 2019STD: Sparse-to-Dense 3D Object Detector for Point Cloudcited by 282
  6. 2020PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentationcited by 275
  7. 2019CoverNet: Multimodal Behavior Prediction using Trajectory Setscited by 259
  8. 2020It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Predictioncited by 236
  9. 2020Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Netcited by 228
  10. 2018Convolutional Social Pooling for Vehicle Trajectory Predictioncited by 221
  11. 2020Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Predictioncited by 218
  12. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 179
  13. 2019Multiple Futures Predictioncited by 178
  14. 2019End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Cloudscited by 171
  15. 2021IntentNet: Learning to Predict Intention from Raw Sensor Datacited by 158
  16. 2019LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Drivingcited by 133
  17. 2016Vehicle Detection from 3D Lidar Using Fully Convolutional Networkcited by 107
  18. 2019Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Datacited by 104
  19. 2020AMVNet: Assertion-based Multi-View Fusion Network for LiDAR Semantic Segmentationcited by 87
  20. 2019Sensor Fusion for Joint 3D Object Detection and Semantic Segmentationcited by 48
  21. 2020KPRNet: Improving projection-based LiDAR semantic segmentationcited by 47
  22. 2020Multi-View Fusion of Sensor Data for Improved Perception and Prediction in Autonomous Drivingcited by 41
  23. 2020RangeRCNN: Towards Fast and Accurate 3D Object Detection with Range Image Representationcited by 37
  24. 2019Learning an Uncertainty-Aware Object Detector for Autonomous Drivingcited by 34
  25. 2020MultiXNet: Multiclass Multistage Multimodal Motion Predictioncited by 31
  26. 2020LaserFlow: Efficient and Probabilistic Object Detection and Motion Forecastingcited by 22
  27. 2020MVLidarNet: Real-Time Multi-Class Scene Understanding for Autonomous Driving Using Multiple Viewscited by 20
  28. 2020LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusioncited by 18
  29. 2020RV-FuseNet: Range View Based Fusion of Time-Series LiDAR Data for Joint 3D Object Detection and Motion Forecastingcited by 8

Built on by

Cited by year

Citing papers per year: 2022: 6; 2023: 8; 2024: 3; 2025: 1.

18 papers.

Sort:

By area:

  1. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
  2. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  3. 2022SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimationcited by 76
  4. 2022VISTA: Boosting 3D Object Detection via Dual Cross-VIew SpaTial Attentioncited by 48
  5. 2022PreTraM: Self-Supervised Pre-training via Connecting Trajectory and Mapcited by 25
  6. 2022RaLiBEV: Radar and LiDAR BEV Fusion Learning for Anchor Box Free Object Detection Systemscited by 23
  7. 2023Pedestrian Crossing Action Recognition and Trajectory Prediction with 3D Human Keypointscited by 12
  8. 2023STTracker: Spatio-Temporal Tracker for 3D Single Object Trackingcited by 10
  9. 2023VT-Former: An Exploratory Study on Vehicle Trajectory Prediction for Highway Surveillance through Graph Isomorphism and Transformercited by 10
  10. 2023MGTR: Multi-Granular Transformer for Motion Prediction with LiDARcited by 9
  11. 2024Joint Perception and Prediction for Autonomous Driving: A Surveycited by 7
  12. 2023Practical Collaborative Perception: A Framework for Asynchronous and Multi-Agent 3D Object Detectioncited by 6
6 more
  1. 2024Robust Long-Range Perception Against Sensor Misalignment in Autonomous Vehiclescited by 5
  2. 2023Trajectory-Prediction with Vision: A Surveycited by 4
  3. 2023CVSformer: Cross-View Synthesis Transformer for Semantic Scene Completioncited by 4
  4. 2023Aligning Bird-Eye View Representation of Point Cloud Sequences using Scene Flowcited by 1
  5. 2025To New Beginnings: A Survey of Unified Perception in Autonomous Vehicle Softwarecited by 1
  6. 2024What Matters in Range View 3D Object Detectioncited by 0