# BEVMOSNet: Multimodal Fusion for BEV Moving Object Segmentation

arXiv: https://arxiv.org/abs/2503.03280 · HTML: https://mlsanity.com/paper/2503.03280/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 40 references that are in the map.
- First posted on arXiv 5 Mar 2025.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework](https://mlsanity.com/paper/2205.13790/) (2022)
- [FUTR3D: A Unified Sensor Fusion Framework for 3D Detection](https://mlsanity.com/paper/2203.10642/) (2022)
- [Predicting Semantic Map Representations from Images using Pyramid Occupancy Networks](https://mlsanity.com/paper/2003.13402/) (2020)
- [CRN: Camera Radar Net for Accurate, Robust, Efficient 3D Perception](https://mlsanity.com/paper/2304.00670/) (2023)
- [Moving Object Segmentation in 3D LiDAR Data: A Learning-based Approach Exploiting Sequential Data](https://mlsanity.com/paper/2105.08971/) (2021)
- [Learning Cross-Modal Deep Representations for Robust Pedestrian Detection](https://mlsanity.com/paper/1704.02431/) (2017)
- [TBP-Former: Learning Temporal Bird's-Eye-View Pyramid for Joint Perception and Prediction in Vision-Centric Autonomous Driving](https://mlsanity.com/paper/2303.09998/) (2023)
- [Probabilistic Oriented Object Detection in Automotive Radar](https://mlsanity.com/paper/2004.05310/) (2020)
- [Spatio-Contextual Deep Network Based Multimodal Pedestrian Detection For Autonomous Driving](https://mlsanity.com/paper/2105.12713/) (2021)
- [PowerBEV: A Powerful Yet Lightweight Framework for Instance Prediction in Bird's-Eye View](https://mlsanity.com/paper/2306.10761/) (2023)
- [RaTrack: Moving Object Detection and Tracking with 4D Radar Point Cloud](https://mlsanity.com/paper/2309.09737/) (2023)
- [InsMOS: Instance-Aware Moving Object Segmentation in LiDAR Data](https://mlsanity.com/paper/2303.03909/) (2023)
- [TiledSoilingNet: Tile-level Soiling Detection on Automotive Surround-view Cameras Using Coverage Metric](https://mlsanity.com/paper/2007.00801/) (2020)
- [LiMoSeg: Real-time Bird's Eye View based LiDAR Motion Segmentation](https://mlsanity.com/paper/2111.04875/) (2021)
- [MambaMOS: LiDAR-based 3D Moving Object Segmentation with Motion-aware State Space Model](https://mlsanity.com/paper/2404.12794/) (2024)
- [Revisiting Modality Imbalance In Multimodal Pedestrian Detection](https://mlsanity.com/paper/2302.12589/) (2023)
- [MotionBEV: Attention-Aware Online LiDAR Moving Object Segmentation with Bird's Eye View based Appearance and Motion Features](https://mlsanity.com/paper/2305.07336/) (2023)
- [Radar Instance Transformer: Reliable Moving Instance Segmentation in Sparse Radar Point Clouds](https://mlsanity.com/paper/2309.16435/) (2023)
- [RadarMOSEVE: A Spatial-Temporal Transformer Network for Radar-Only Moving Object Segmentation and Ego-Velocity Estimation](https://mlsanity.com/paper/2402.14380/) (2024)
- [BEV-MODNet: Monocular Camera based Bird's Eye View Moving Object Detection for Autonomous Driving](https://mlsanity.com/paper/2107.04937/) (2021)
- [Exploiting Sparsity in Automotive Radar Object Detection Networks](https://mlsanity.com/paper/2308.07748/) (2023)
- [Fisheye Camera and Ultrasonic Sensor Fusion For Near-Field Obstacle Perception in Bird's-Eye-View](https://mlsanity.com/paper/2402.00637/) (2024)

## Abstract (arXiv)

Accurate motion understanding of the dynamic objects within the scene in bird's-eye-view (BEV) is critical to ensure a reliable obstacle avoidance system and smooth path planning for autonomous vehicles. However, this task has received relatively limited exploration when compared to object detection and segmentation with only a few recent vision-based approaches presenting preliminary findings that significantly deteriorate in low-light, nighttime, and adverse weather conditions such as rain. Conversely, LiDAR and radar sensors remain almost unaffected in these scenarios, and radar provides key velocity information of the objects. Therefore, we introduce BEVMOSNet, to our knowledge, the first end-to-end multimodal fusion leveraging cameras, LiDAR, and radar to precisely predict the moving objects in BEV. In addition, we perform a deeper analysis to find out the optimal strategy for deformable cross-attention-guided sensor fusion for cross-sensor knowledge sharing in BEV. While evaluating BEVMOSNet on the nuScenes dataset, we show an overall improvement in IoU score of 36.59% compared to the vision-based unimodal baseline BEV-MoSeg (Sigatapu et al., 2023), and 2.35% compared to the multimodel SimpleBEV (Harley et al., 2022), extended for the motion segmentation task, establishing this method as the state-of-the-art in BEV motion segmentation.