# Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completion

arXiv: https://arxiv.org/abs/2203.09780 · HTML: https://mlsanity.com/paper/2203.09780/

## Key facts

- Cited by 82 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 50 references that are in the map.
- First posted on arXiv 18 Mar 2022.

## Builds on (most cited first, 30 of 45)

- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [PointPainting: Sequential Fusion for 3D Object Detection](https://mlsanity.com/paper/1911.10150/) (2019)
- [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020)
- [Selective Kernel Networks](https://mlsanity.com/paper/1903.06586/) (2019)
- [From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network](https://mlsanity.com/paper/1907.03670/) (2019)
- [Joint 3D Proposal Generation and Object Detection from View Aggregation](https://mlsanity.com/paper/1712.02294/) (2017)
- [STD: Sparse-to-Dense 3D Object Detector for Point Cloud](https://mlsanity.com/paper/1907.10471/) (2019)
- [Deep Continuous Fusion for Multi-Sensor 3D Object Detection](https://mlsanity.com/paper/2012.10992/) (2020)
- [Voxel Transformer for 3D Object Detection](https://mlsanity.com/paper/2109.02497/) (2021)
- [Multi-Task Multi-Sensor Fusion for 3D Object Detection](https://mlsanity.com/paper/2012.12397/) (2020)
- [Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/1906.06310/) (2019)
- [3D-CVF: Generating Joint Camera and LiDAR Features Using Cross-View Spatial Feature Fusion for 3D Object Detection](https://mlsanity.com/paper/2004.12636/) (2020)
- [PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation](https://mlsanity.com/paper/1711.10871/) (2017)
- [EPNet: Enhancing Point Features with Image Semantics for 3D Object Detection](https://mlsanity.com/paper/2007.08856/) (2020)
- [MVX-Net: Multimodal VoxelNet for 3D Object Detection](https://mlsanity.com/paper/1904.01649/) (2019)
- [CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detection](https://mlsanity.com/paper/2009.00784/) (2020)
- [PENet: Towards Precise and Efficient Image Guided Depth Completion](https://mlsanity.com/paper/2103.00783/) (2021)
- [RangeDet:In Defense of Range View for LiDAR-based 3D Object Detection](https://mlsanity.com/paper/2103.10039/) (2021)
- [LiDAR R-CNN: An Efficient and Universal 3D Object Detector](https://mlsanity.com/paper/2103.15297/) (2021)
- [Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection](https://mlsanity.com/paper/1903.01864/) (2019)
- [Fast Point R-CNN](https://mlsanity.com/paper/1908.02990/) (2019)
- [SE-SSD: Self-Ensembling Single-Stage Object Detector From Point Cloud](https://mlsanity.com/paper/2104.09804/) (2021)

## Built on by (most cited first, 30 of 82)

- [Virtual Sparse Convolution for Multimodal 3D Object Detection](https://mlsanity.com/paper/2303.02314/) (2023)
- [LoGoNet: Towards Accurate 3D Object Detection with Local-to-Global Cross-Modal Fusion](https://mlsanity.com/paper/2303.03595/) (2023)
- [GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Clouds](https://mlsanity.com/paper/2212.03010/) (2022)
- [Transformation-Equivariant 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/2211.11962/) (2022)
- [Graph R-CNN: Towards Accurate 3D Object Detection with Semantic-Decorated Local Graph](https://mlsanity.com/paper/2208.03624/) (2022)
- [EA-LSS: Edge-aware Lift-splat-shot Framework for 3D BEV Object Detection](https://mlsanity.com/paper/2303.17895/) (2023)
- [Learning Occupancy for Monocular 3D Object Detection](https://mlsanity.com/paper/2305.15694/) (2023)
- [GraphAlign: Enhancing Accurate Feature Alignment by Graph matching for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2310.08261/) (2023)
- [Detecting As Labeling: Rethinking LiDAR-camera Fusion in 3D Object Detection](https://mlsanity.com/paper/2311.07152/) (2023)
- [Depth Prompting for Sensor-Agnostic Depth Estimation](https://mlsanity.com/paper/2405.11867/) (2024)
- [PVT-SSD: Single-Stage 3D Object Detector with Point-Voxel Transformer](https://mlsanity.com/paper/2305.06621/) (2023)
- [MonoLSS: Learnable Sample Selection For Monocular 3D Detection](https://mlsanity.com/paper/2312.14474/) (2023)
- [Distilling Monocular Foundation Model for Fine-grained Depth Completion](https://mlsanity.com/paper/2503.16970/) (2025)
- [3D-QueryIS: A Query-based Framework for 3D Instance Segmentation](https://mlsanity.com/paper/2211.09375/) (2022)
- [SupFusion: Supervised LiDAR-Camera Fusion for 3D Object Detection](https://mlsanity.com/paper/2309.07084/) (2023)
- [VoxelNextFusion: A Simple, Unified and Effective Voxel Fusion Framework for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2401.02702/) (2024)
- [Unsupervised Adaptation from Repeated Traversals for Autonomous Driving](https://mlsanity.com/paper/2303.15286/) (2023)
- [NeurOCS: Neural NOCS Supervision for Monocular 3D Object Localization](https://mlsanity.com/paper/2305.17763/) (2023)
- [MoDAR: Using Motion Forecasting for 3D Object Detection in Point Cloud Sequences](https://mlsanity.com/paper/2306.03206/) (2023)
- [Commonsense Prototype for Outdoor Unsupervised 3D Object Detection](https://mlsanity.com/paper/2404.16493/) (2024)
- [PG-RCNN: Semantic Surface Point Generation for 3D Object Detection](https://mlsanity.com/paper/2307.12637/) (2023)
- [Recent Advances in Multi-modal 3D Intelligence: A Comprehensive Survey and Evaluation](https://mlsanity.com/paper/2310.15676/) (2023)
- [Multi-Sem Fusion: Multimodal Semantic Fusion for 3D Object Detection](https://mlsanity.com/paper/2212.05265/) (2022)
- [Unleash the Potential of Image Branch for Cross-modal 3D Object Detection](https://mlsanity.com/paper/2301.09077/) (2023)
- [Sunshine to Rainstorm: Cross-Weather Knowledge Distillation for Robust 3D Object Detection](https://mlsanity.com/paper/2402.18493/) (2024)
- [Eliminating Cross-modal Conflicts in BEV Space for LiDAR-Camera 3D Object Detection](https://mlsanity.com/paper/2403.07372/) (2024)
- [LiDPM: Rethinking Point Diffusion for Lidar Scene Completion](https://mlsanity.com/paper/2504.17791/) (2025)
- [Sparse Dense Fusion for 3D Object Detection](https://mlsanity.com/paper/2304.04179/) (2023)
- [Semi-supervised 3D Object Detection with PatchTeacher and PillarMix](https://mlsanity.com/paper/2407.09787/) (2024)
- [RGB Guided ToF Imaging System: A Survey of Deep Learning-based Methods](https://mlsanity.com/paper/2405.10357/) (2024)

## Abstract (arXiv)

Current LiDAR-only 3D detection methods inevitably suffer from the sparsity of point clouds. Many multi-modal methods are proposed to alleviate this issue, while different representations of images and point clouds make it difficult to fuse them, resulting in suboptimal performance. In this paper, we present a novel multi-modal framework SFD (Sparse Fuse Dense), which utilizes pseudo point clouds generated from depth completion to tackle the issues mentioned above. Different from prior works, we propose a new RoI fusion strategy 3D-GAF (3D Grid-wise Attentive Fusion) to make fuller use of information from different types of point clouds. Specifically, 3D-GAF fuses 3D RoI features from the couple of point clouds in a grid-wise attentive way, which is more fine-grained and more precise. In addition, we propose a SynAugment (Synchronized Augmentation) to enable our multi-modal framework to utilize all data augmentation approaches tailored to LiDAR-only methods. Lastly, we customize an effective and efficient feature extractor CPConv (Color Point Convolution) for pseudo point clouds. It can explore 2D image features and 3D geometric features of pseudo point clouds simultaneously. Our method holds the highest entry on the KITTI car 3D object detection leaderboard, demonstrating the effectiveness of our SFD. Codes are available at https://github.com/LittlePey/SFD.