mlsanityData to 14 Oct 2026

Virtual Sparse Convolution for Multimodal 3D Object Detection

arXiv 2303.02314 ↗First posted 4 Mar 2023cs.CVcited by 81

Not on the map. Why?

Abstract (arXiv)

Recently, virtual/pseudo-point-based 3D object detection that seamlessly fuses RGB images and LiDAR data by depth completion has gained great attention. However, virtual points generated from an image are very dense, introducing a huge amount of redundant computation during detection. Meanwhile, noises brought by inaccurate depth completion significantly degrade detection precision. This paper proposes a fast yet effective backbone, termed VirConvNet, based on a new operator VirConv (Virtual Sparse Convolution), for virtual-point-based 3D object detection. VirConv consists of two key designs: (1) StVD (Stochastic Voxel Discard) and (2) NRConv (Noise-Resistant Submanifold Convolution). StVD alleviates the computation problem by discarding large amounts of nearby redundant voxels. NRConv tackles the noise problem by encoding voxel features in both 2D image and 3D LiDAR space. By integrating VirConv, we first develop an efficient pipeline VirConv-L based on an early fusion design. Then, we build a high-precision pipeline VirConv-T based on a transformed refinement scheme. Finally, we develop a semi-supervised pipeline VirConv-S based on a pseudo-label framework. On the KITTI car 3D detection test leaderboard, our VirConv-L achieves 85% AP with a fast running speed of 56ms. Our VirConv-T and VirConv-S attains a high-precision of 86.3% and 87.2% AP, and currently rank 2nd and 1st, respectively. The code is available at https://github.com/hailanyi/VirConv.

Key facts. Cited by 81 papers in the map (data through 14 Oct 2026). Builds on 41 of its 51 references that are in the map. First posted on arXiv 4 Mar 2023.

Builds on

41 of 51 references.

References · 41

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  2. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  3. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  4. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 941
  5. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 891
  6. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
  7. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 748
  8. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 724
  9. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  10. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 585
  11. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 466
  12. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
29 more
  1. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  2. 2020Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detectioncited by 410
  3. 2019From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Networkcited by 368
  4. 2017Joint 3D Proposal Generation and Object Detection from View Aggregationcited by 326
  5. 2019STD: Sparse-to-Dense 3D Object Detector for Point Cloudcited by 282
  6. 2020Multi-Task Multi-Sensor Fusion for 3D Object Detectioncited by 203
  7. 2022FUTR3D: A Unified Sensor Fusion Framework for 3D Detectioncited by 194
  8. 20203D-CVF: Generating Joint Camera and LiDAR Features Using Cross-View Spatial Feature Fusion for 3D Object Detectioncited by 176
  9. 2021Multimodal Virtual Point 3D Detectioncited by 166
  10. 2019MVX-Net: Multimodal VoxelNet for 3D Object Detectioncited by 163
  11. 2021ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detectioncited by 159
  12. 2021PENet: Towards Precise and Efficient Image Guided Depth Completioncited by 146
  13. 2022Focal Sparse Convolutional Networks for 3D Object Detectioncited by 142
  14. 2019IoU Loss for 2D/3D Object Detectioncited by 128
  15. 2021SE-SSD: Self-Ensembling Single-Stage Object Detector From Point Cloudcited by 119
  16. 2021Improving 3D Object Detection with Channel-wise Transformercited by 108
  17. 2019SESS: Self-Ensembling Semi-Supervised 3D Object Detectioncited by 89
  18. 20203DIoUMatch: Leveraging IoU Prediction for Semi-Supervised 3D Object Detectioncited by 89
  19. 2021Score-Based Point Cloud Denoisingcited by 83
  20. 2022Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completioncited by 82
  21. 2021Pyramid R-CNN: Towards Better Performance and Adaptability for 3D Object Detectioncited by 79
  22. 2019Bi-Directional Cascade Network for Perceptual Edge Detectioncited by 72
  23. 2021Behind the Curtain: Learning Occluded Shapes for 3D Object Detectioncited by 58
  24. 2021VPFNet: Improving 3D Object Detection with Virtual Point based LiDAR and Stereo Data Fusioncited by 48
  25. 2022Graph R-CNN: Towards Accurate 3D Object Detection with Semantic-Decorated Local Graphcited by 42
  26. 2019Monocular Plan View Networks for Autonomous Drivingcited by 37
  27. 2018BirdNet: a 3D Object Detection Framework from LiDAR informationcited by 33
  28. 2019CNN-based Lidar Point Cloud De-Noising in Adverse Weathercited by 33
  29. 2021Semi-supervised 3D Object Detection via Adaptive Pseudo-Labelingcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 11; 2024: 33; 2025: 26; 2026: 11.

81 papers.

Sort:

By area:

  1. 2023UniTR: A Unified and Efficient Multi-Modal Transformer for Bird's-Eye-View Representationcited by 74
  2. 2023Generative AI meets 3D: A Survey on Text-to-3D in AIGC Eracited by 61
  3. 2024IS-Fusion: Instance-Scene Collaborative Fusion for Multimodal 3D Object Detectioncited by 57
  4. 2023Detecting As Labeling: Rethinking LiDAR-camera Fusion in 3D Object Detectioncited by 28
  5. 2024OV-Uni3DETR: Towards Unified Open-Vocabulary 3D Object Detection via Cycle-Modality Propagationcited by 19
  6. 2024V2X-R: Cooperative LiDAR-4D Radar Fusion with Denoising Diffusion for 3D Object Detectioncited by 18
  7. 2024MCTrack: A Unified 3D Multi-Object Tracking Framework for Autonomous Drivingcited by 17
  8. 2024VoxelNextFusion: A Simple, Unified and Effective Voxel Fusion Framework for Multi-Modal 3D Object Detectioncited by 14
  9. 2025Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directionscited by 14
  10. 2024Commonsense Prototype for Outdoor Unsupervised 3D Object Detectioncited by 13
  11. 2025DriveGEN: Generalized and Robust 3D Detection in Driving via Controllable Text-to-Image Diffusion Generationcited by 13
  12. 2024Sunshine to Rainstorm: Cross-Weather Knowledge Distillation for Robust 3D Object Detectioncited by 11
38 more
  1. 2024MonoTAKD: Teaching Assistant Knowledge Distillation for Monocular 3D Object Detectioncited by 10
  2. 2024PointSeg: A Training-Free Paradigm for 3D Scene Segmentation via Foundation Modelscited by 9
  3. 2024RobMOT: Robust 3D Multi-Object Tracking by Observational Noise and State Estimation Drift Mitigation on LiDAR PointCloudcited by 9
  4. 2023PeP: a Point enhanced Painting method for unified point cloud taskscited by 7
  5. 2024Density-guided Translator Boosts Synthetic-to-Real Unsupervised Domain Adaptive Segmentation of 3D Point Cloudscited by 7
  6. 2025Learning to Detect Objects from Multi-Agent LiDAR Scans without Manual Labelscited by 7
  7. 2024BiTrack: Bidirectional Offline 3D Multi-Object Tracking Using Camera-LiDAR Datacited by 6
  8. 2024FSF-Net: Enhance 4D Occupancy Forecasting with Coarse BEV Scene Flow for Autonomous Drivingcited by 6
  9. 2025SP3D: Boosting Sparsely-Supervised 3D Object Detection via Accurate Cross-Modal Semantic Promptscited by 6
  10. 2023Multi-Modal 3D Object Detection by Box Matchingcited by 4
  11. 2024Improving Robustness of LiDAR-Camera Fusion Model against Weather Corruption from Fusion Strategy Perspectivecited by 4
  12. 2024RePLAy: Remove Projective LiDAR Depthmap Artifacts via Exploiting Epipolar Geometrycited by 4
  13. 2025GATE3D: Generalized Attention-based Task-synergized Estimation in 3D*cited by 4
  14. 2024GraphRelate3D: Context-Dependent 3D Object Detection with Inter-Object Relationship Graphscited by 3
  15. 2024LEROjD: Lidar Extended Radar-Only Object Detectioncited by 3
  16. 2024PhantomLiDAR: Cross-modality Signal Injection Attacks against LiDARcited by 3
  17. 2025HybridTrack: A Hybrid Approach for Robust Multi-Object Trackingcited by 3
  18. 2025Rethinking Backbone Design for Lightweight 3D Object Detection in LiDARcited by 3
  19. 2023LiDAR-BEVMTN: Real-Time LiDAR Bird's-Eye View Multi-Task Perception Network for Autonomous Drivingcited by 2
  20. 2023Progressive Classifier and Feature Extractor Adaptation for Unsupervised Domain Adaptation on Point Cloudscited by 2
  21. 2024Vision Foundation Model Driven Foreground-Aware Pseudo-LiDAR Generation for Monocular 3D Object Detectioncited by 2
  22. 2024Fully Test-Time Adaptation for Monocular 3D Object Detectioncited by 2
  23. 2024CrossTracker: Robust Multi-modal 3D Multi-Object Tracking via Cross Correctioncited by 2
  24. 2025Beyond Atoms: Enhancing Molecular Pretrained Representations with 3D Space Modelingcited by 2
  25. 2025A Multimodal Hybrid Late-Cascade Fusion Network for Enhanced 3D Object Detectioncited by 2
  26. 2025Multistream Network for LiDAR and Camera-based 3D Object Detection in Outdoor Scenescited by 2
  27. 2025CMF-IoU: Multi-Stage Cross-Modal Fusion 3D Object Detection with IoU Joint Predictioncited by 2
  28. 2025SFGFusion: Surface Fitting Guided 3D Object Detection with 4D Radar and Camera Fusioncited by 2
  29. 2026Offline-Poly: A Polyhedral Framework For Offline 3D Multi-Object Trackingcited by 2
  30. 2023What You See Is What You Detect: Towards better Object Densification in 3D detectioncited by 1
  31. 2024Leveraging Anchor-based LiDAR 3D Object Detection via Point Assisted Sample Selectioncited by 1
  32. 2024Temporal Enhanced Floating Car Observerscited by 1
  33. 2024Cross-Cluster Shifting for Efficient and Effective 3D Object Detection in Autonomous Drivingcited by 1
  34. 2024MonoDETRNext: Next-Generation Accurate and Efficient Monocular 3D Object Detectorcited by 1
  35. 2024TraIL-Det: Transformation-Invariant Local Feature Networks for 3D LiDAR Object Detection with Unsupervised Pre-Trainingcited by 1
  36. 2024One Homography is All You Need: IMM-based Joint Homography and Multiple Object State Estimationcited by 1
  37. 2024FSMDet: Vision-guided feature diffusion for fully sparse 3D detectorcited by 1
  38. 2025Towards Accurate State Estimation: Motion Dynamics Kalman Filter for 3D Multi-Object Trackingcited by 1

50 most cited of 81 All 81 (JSON)