mlsanityData to 7 Oct 2026

PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation

arXiv 1711.10871 ↗First posted 29 Nov 2017cs.CVcited by 170

Not on the map. Why?

Abstract (arXiv)

We present PointFusion, a generic 3D object detection method that leverages both image and 3D point cloud information. Unlike existing methods that either use multi-stage pipelines or hold sensor and dataset-specific assumptions, PointFusion is conceptually simple and application-agnostic. The image data and the raw point cloud data are independently processed by a CNN and a PointNet architecture, respectively. The resulting outputs are then combined by a novel fusion network, which predicts multiple 3D box hypotheses and their confidences, using the input 3D points as spatial anchors. We evaluate PointFusion on two distinctive datasets: the KITTI dataset that features driving scenes captured with a lidar-camera setup, and the SUN-RGBD dataset that captures indoor environments with RGB-D cameras. Our model is the first one that is able to perform better or on-par with the state-of-the-art on these diverse datasets without any dataset-specific model tuning.

Key facts. Cited by 170 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 32; 2022: 52; 2023: 37; 2024: 16; 2025: 24; 2026: 9.

170 papers.

Sort:

By area:

  1. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  2. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  3. 2022Open-vocabulary Queryable Scene Representations for Real World Planningcited by 159
  4. 2021FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimationcited by 144
  5. 2021Offboard 3D Object Detection from Point Cloud Sequencescited by 144
  6. 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
  7. 2022Vision-Centric BEV Perception: A Surveycited by 126
  8. 2021SAT: 2D Semantics Assisted Training for 3D Visual Groundingcited by 124
  9. 2021DualPoseNet: Category-level 6D Object Pose and Size Estimation Using Dual Pose Network with Refined Learning of Pose Consistencycited by 103
  10. 2021Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attackscited by 101
  11. 2021FusionPainting: Multimodal Fusion with Adaptive Attention for 3D Object Detectioncited by 92
  12. 2022Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completioncited by 82
38 more
  1. 2022FS6D: Few-Shot 6D Pose Estimation of Novel Objectscited by 81
  2. 2023LoGoNet: Towards Accurate 3D Object Detection with Local-to-Global Cross-Modal Fusioncited by 80
  3. 2022Multi-modal Sensor Fusion for Auto Driving Perception: A Surveycited by 73
  4. 2022X-Trans2Cap: Cross-Modal Knowledge Transfer using Transformer for 3D Dense Captioningcited by 70
  5. 2021Multi-Modal 3D Object Detection in Autonomous Driving: a Surveycited by 63
  6. 2023Grounding 3D Object Affordance from 2D Interactions in Imagescited by 62
  7. 2021Object DGCNN: 3D Object Detection using Dynamic Graphscited by 58
  8. 2024IS-Fusion: Instance-Scene Collaborative Fusion for Multimodal 3D Object Detectioncited by 57
  9. 2021CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimationcited by 56
  10. 2023PiMAE: Point Cloud and Image Interactive Masked Autoencoders for 3D Object Detectioncited by 53
  11. 2021EPNet++: Cascade Bi-directional Fusion for Multi-Modal 3D Object Detectioncited by 49
  12. 20214D-Net for Learned Multi-Modal Alignmentcited by 44
  13. 2022OVE6D: Object Viewpoint Encoding for Depth-based 6D Object Pose Estimationcited by 43
  14. 2023VL-SAT: Visual-Linguistic Semantics Assisted Training for 3D Semantic Scene Graph Prediction in Point Cloudcited by 42
  15. 2022Look Around and Refer: 2D Synthetic Semantics Knowledge Distillation for 3D Visual Groundingcited by 41
  16. 2022Towards Long-Tailed 3D Detectioncited by 40
  17. 2021M3DSSD: Monocular 3D Single Stage Object Detectorcited by 39
  18. 2025Multimodal Fusion and Vision-Language Models: A Survey for Robot Visioncited by 37
  19. 2022CAT-Det: Contrastively Augmented Transformer for Multi-modal 3D Object Detectioncited by 32
  20. 2022DPODv2: Dense Correspondence-Based 6 DoF Pose Estimationcited by 32
  21. 2023AutoFed: Heterogeneity-Aware Federated Multimodal Learning for Robust Autonomous Drivingcited by 32
  22. 2023SE(3) Diffusion Model-based Point Cloud Registration for Robust 6D Object Pose Estimationcited by 32
  23. 2022HRFuser: A Multi-resolution Sensor Fusion Architecture for 2D Object Detectioncited by 30
  24. 2024MatchU: Matching Unseen Objects for 6D Pose Estimation from RGB-D Imagescited by 28
  25. 2022Homogeneous Multi-modal Feature Fusion and Interaction for 3D Object Detectioncited by 26
  26. 2022A Closer Look at Invariances in Self-supervised Pre-training for 3D Visioncited by 24
  27. 2022Open-Vocabulary 3D Detection via Image-level Class and Debiased Cross-modal Contrastive Learningcited by 23
  28. 2022Uni6D: A Unified CNN Framework without Projection Breakdown for 6D Pose Estimationcited by 22
  29. 2023See More and Know More: Zero-shot Point Cloud Segmentation via Multi-modal Visual Datacited by 21
  30. 2024Learning 2D Invariant Affordance Knowledge for 3D Affordance Groundingcited by 21
  31. 2025InteractVLM: 3D Interaction Reasoning from 2D Foundational Modelscited by 21
  32. 20216D-ViT: Category-Level 6D Object Pose Estimation via Transformer-based Instance Representation Learningcited by 20
  33. 2025SuperPC: A Single Diffusion Model for Point Cloud Completion, Upsampling, Denoising, and Colorizationcited by 20
  34. 2022Bridged Transformer for Vision and Point Cloud 3D Object Detectioncited by 19
  35. 2021Revisiting 3D Object Detection From an Egocentric Perspectivecited by 18
  36. 2021A Simple and Efficient Multi-task Network for 3D Object Detection and Road Understandingcited by 17
  37. 2021PR-GCN: A Deep Graph Convolutional Network with Point Refinement for 6D Pose Estimationcited by 17
  38. 2021Sparse Steerable Convolutions: An Efficient Learning of SE(3)-Equivariant Features for Estimation and Tracking of Object Poses in 3D Spacecited by 17

50 most cited of 170 All 170 (JSON)