mlsanityData to 14 Oct 2026

Multimodal and Multiview Deep Fusion for Autonomous Marine Navigation

arXiv 2505.01615 ↗First posted 2 May 2025cs.CV cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

We propose a cross attention transformer based method for multimodal sensor fusion to build a birds eye view of a vessels surroundings supporting safer autonomous marine navigation. The model deeply fuses multiview RGB and long wave infrared images with sparse LiDAR point clouds. Training also integrates X band radar and electronic chart data to inform predictions. The resulting view provides a detailed reliable scene representation improving navigational accuracy and robustness. Real world sea trials confirm the methods effectiveness even in adverse weather and complex maritime settings.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 55 of its 75 references that are in the map. First posted on arXiv 2 May 2025.

Builds on

55 of 75 references.

References · 55

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  4. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  5. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  6. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  7. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  8. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  9. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
  10. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 891
  11. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
  12. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 724
38 more
  1. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 572
  2. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  3. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 457
  4. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  5. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 447
  6. 2019Are Transformers universal approximators of sequence-to-sequence functions?cited by 406
  7. 2018Rethinking ImageNet Pre-trainingcited by 377
  8. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 368
  9. 2019PIXOR: Real-time 3D Object Detection from Point Cloudscited by 358
  10. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 336
  11. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 327
  12. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  13. 2021HDMapNet: An Online HD Map Construction and Evaluation Frameworkcited by 281
  14. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  15. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 249
  16. 2021Voxel Transformer for 3D Object Detectioncited by 247
  17. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  18. 2022DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detectioncited by 224
  19. 2020SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimationcited by 204
  20. 2022Unifying Voxel-based Representation with Transformer for 3D Object Detectioncited by 202
  21. 2021Embracing Single Stride 3D Object Detector with Sparse Transformercited by 193
  22. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  23. 2019Cross-view Semantic Segmentation for Sensing Surroundingscited by 174
  24. 2021Multimodal Virtual Point 3D Detectioncited by 166
  25. 2022Simple-BEV: What Really Matters for Multi-Sensor BEV Perception?cited by 128
  26. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  27. 2021Translating Images into Mapscited by 115
  28. 20183D-LaneNet: End-to-End 3D Multiple Lane Detectioncited by 107
  29. 2020DSGN: Deep Stereo Geometry Network for 3D Object Detectioncited by 101
  30. 2021AFDetV2: Rethinking the Necessity of the Second Stage for Object Detection from Point Cloudscited by 88
  31. 2021Pyramid R-CNN: Towards Better Performance and Adaptability for 3D Object Detectioncited by 79
  32. 2020HDNET: Exploiting HD Maps for 3D Object Detectioncited by 78
  33. 2020A Sim2Real Deep Learning Approach for the Transformation of Images from Multiple Vehicle-Mounted Cameras to a Semantically Segmented Image in Bird's Eye Viewcited by 72
  34. 2021LIGA-Stereo: Learning LiDAR Geometry Aware Representations for Stereo-based 3D Detectorcited by 70
  35. 2022AutoAlign: Pixel-Instance Feature Aggregation for Multi-Modal 3D Object Detectioncited by 70
  36. 2022MPPNet: Multi-Frame Feature Intertwining with Proxy Points for 3D Temporal Object Detectioncited by 65
  37. 2018Complex-YOLO: Real-time 3D Object Detection on Point Cloudscited by 57
  38. 2020FISHING Net: Future Inference of Semantic Heatmaps In Gridscited by 55

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026The Coastline as a Structural Constraint: Harnessing Scene Geometry for Autonomous Surface Vessel Localizationcited by 0