mlsanityData to 7 Oct 2026

MV2DFusion: Leveraging Modality-Specific Object Semantics for Multi-Modal 3D Detection

arXiv 2408.05945 ↗First posted 12 Aug 2024cs.CVcited by 23

Not on the map. Why?

Abstract (arXiv)

The rise of autonomous vehicles has significantly increased the demand for robust 3D object detection systems. While cameras and LiDAR sensors each offer unique advantages--cameras provide rich texture information and LiDAR offers precise 3D spatial data--relying on a single modality often leads to performance limitations. This paper introduces MV2DFusion, a multi-modal detection framework that integrates the strengths of both worlds through an advanced query-based fusion mechanism. By introducing an image query generator to align with image-specific attributes and a point cloud query generator, MV2DFusion effectively combines modality-specific object semantics without biasing toward one single modality. Then the sparse fusion process can be accomplished based on the valuable object semantics, ensuring efficient and accurate object detection across various scenarios. Our framework's flexibility allows it to integrate with any image and point cloud-based detectors, showcasing its adaptability and potential for future advancements. Extensive evaluations on the nuScenes and Argoverse2 datasets demonstrate that MV2DFusion achieves state-of-the-art performance, particularly excelling in long-range detection scenarios.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 70 of its 79 references that are in the map. First posted on arXiv 12 Aug 2024.

Builds on

70 of 79 references.

References · 70

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2017Mask R-CNNcited by 7,259
  7. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  8. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  9. 2022A ConvNet for the 2020scited by 3,545
  10. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  11. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  12. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
38 more
  1. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  2. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  3. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  4. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  5. 2019Objects as Pointscited by 1,172
  6. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  7. 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,064
  8. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  9. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 939
  10. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 890
  11. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  12. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  13. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
  14. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 701
  15. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
  16. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  17. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  18. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  19. 2022DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingcited by 486
  20. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 464
  21. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 455
  22. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  23. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  24. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  25. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  26. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 340
  27. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 334
  28. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 313
  29. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  30. 2022DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detectioncited by 224
  31. 2023VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Trackingcited by 221
  32. 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
  33. 2022Unifying Voxel-based Representation with Transformer for 3D Object Detectioncited by 202
  34. 2021Multimodal Virtual Point 3D Detectioncited by 166
  35. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 161
  36. 2022PolarFormer: Multi-camera 3D Object Detection with Polar Transformercited by 147
  37. 2021RangeDet:In Defense of Range View for LiDAR-based 3D Object Detectioncited by 140
  38. 2022DeepInteraction: 3D Object Detection via Modality Interactioncited by 125

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 8; 2026: 13.

23 papers.

Sort:

By area:

  1. 2025Distilling Multi-modal Large Language Models for Autonomous Drivingcited by 46
  2. 2025Multimodal Fusion and Vision-Language Models: A Survey for Robot Visioncited by 37
  3. 2025Bridging Perspectives: Foundation Model Guided BEV Maps for 3D Object Detection and Trackingcited by 9
  4. 2025RoCA: Robust Cross-Domain End-to-End Autonomous Drivingcited by 8
  5. 2024Timealign: A multi-modal object detection method for time misalignment fusing in autonomous drivingcited by 3
  6. 2025Rethinking the Spatio-Temporal Alignment of End-to-End 3D Perceptioncited by 3
  7. 2026CCF: Complementary Collaborative Fusion for Domain Generalized Multi-Modal 3D Object Detectioncited by 3
  8. 2024EVT: Efficient View Transformation for Multi-Modal 3D Object Detectioncited by 2
  9. 2025DGFusion: Dual-guided Fusion for Robust Multi-Modal 3D Object Detectioncited by 2
  10. 2026Offline-Poly: A Polyhedral Framework For Offline 3D Multi-Object Trackingcited by 2
  11. 2026DSERT-RoLL: Robust Multi-Modal Perception for Diverse Driving Conditions with Stereo Event-RGB-Thermal Cameras, 4D Radar, and Dual-LiDARcited by 2
  12. 2026MambaFusion: Adaptive State-Space Fusion for Multimodal 3D Object Detectioncited by 1
11 more
  1. 2026Fusion-Poly: A Polyhedral Framework Based on Spatial-Temporal Fusion for 3D Multi-Object Trackingcited by 1
  2. 2026FOMO-3D: Using Vision Foundation Models for Long-Tailed 3D Object Detectioncited by 1
  3. 2025An Efficient LiDAR-Camera Fusion Network for Multi-Class 3D Dynamic Object Detection and Trajectory Predictioncited by 0
  4. 2025SCAFusion: A Multimodal 3D Detection Framework for Small Object Detection in Lunar Surface Explorationcited by 0
  5. 2026Fusion4CA: Boosting 3D Object Detection via Comprehensive Image Exploitationcited by 0
  6. 2026RayMamba: Ray-Aligned Serialization for Long-Range 3D Object Detectioncited by 0
  7. 2026Control Your Queries: Heterogeneous Query Interaction for Camera-Radar Fusioncited by 0
  8. 2026Hyperbolic Distillation: Geometry-Guided Cross-Modal Transfer for Robust 3D Object Detectioncited by 0
  9. 2026GraphBEV++: Multi-Modal Feature Alignment for Autonomous Drivingcited by 0
  10. 2026GaussianFusion: Unified 3D Gaussian Representation for Multi-Modal Fusion Perceptioncited by 0
  11. 2026MatchFusion: Explicit-Implicit Instance Matching for Spatio-Temporal Multimodal Autonomous Drivingcited by 0