mlsanityData to 7 Oct 2026

Frustum PointNets for 3D Object Detection from RGB-D Data

arXiv 1711.08488 ↗First posted 22 Nov 2017cs.CVcited by 640

Not on the map. Why?

Abstract (arXiv)

In this work, we study 3D object detection from RGB-D data in both indoor and outdoor scenes. While previous methods focus on images or 3D voxels, often obscuring natural 3D patterns and invariances of 3D data, we directly operate on raw point clouds by popping up RGB-D scans. However, a key challenge of this approach is how to efficiently localize objects in point clouds of large-scale scenes (region proposal). Instead of solely relying on 3D proposals, our method leverages both mature 2D object detectors and advanced 3D deep learning for object localization, achieving efficiency as well as high recall for even small objects. Benefited from learning directly in raw point clouds, our method is also able to precisely estimate 3D bounding boxes even under strong occlusion or with very sparse points. Evaluated on KITTI and SUN RGB-D 3D detection benchmarks, our method outperforms the state of the art by remarkable margins while having real-time capability.

Key facts. Cited by 640 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 173; 2022: 181; 2023: 133; 2024: 71; 2025: 65; 2026: 17.

640 papers.

Sort:

By area:

  1. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  2. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  3. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 464
  4. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 461
  5. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
  6. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 334
  7. 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329
  8. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
  9. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  10. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
  11. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 287
  12. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 248
38 more
  1. 2021Voxel Transformer for 3D Object Detectioncited by 247
  2. 2021TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditionscited by 243
  3. 2021PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detectioncited by 232
  4. 2022DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detectioncited by 224
  5. 2021One Million Scenes for Autonomous Driving: ONCE Datasetcited by 218
  6. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
  7. 2021Group-Free 3D Object Detection via Transformerscited by 215
  8. 2021PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Cloudscited by 211
  9. 2022SoftGroup for 3D Instance Segmentation on Point Cloudscited by 197
  10. 2022FUTR3D: A Unified Sensor Fusion Framework for 3D Detectioncited by 194
  11. 2021Embracing Single Stride 3D Object Detector with Sparse Transformercited by 193
  12. 2021Objects are Different: Flexible Monocular 3D Object Detectioncited by 184
  13. 2021Multimodal Virtual Point 3D Detectioncited by 166
  14. 2022Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wildcited by 159
  15. 2021ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detectioncited by 158
  16. 2021ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detectioncited by 157
  17. 2023V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perceptioncited by 155
  18. 2021FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimationcited by 144
  19. 2021Offboard 3D Object Detection from Point Cloud Sequencescited by 144
  20. 2022Focal Sparse Convolutional Networks for 3D Object Detectioncited by 142
  21. 2021RangeDet:In Defense of Range View for LiDAR-based 3D Object Detectioncited by 140
  22. 2021Delving into Localization Errors for Monocular 3D Object Detectioncited by 138
  23. 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
  24. 2021LiDAR R-CNN: An Efficient and Universal 3D Object Detectorcited by 129
  25. 2022Vision-Centric BEV Perception: A Surveycited by 126
  26. 2021SAT: 2D Semantics Assisted Training for 3D Visual Groundingcited by 124
  27. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  28. 2021FS-Net: Fast Shape-based Network for Category-Level 6D Object Pose Estimation with Decoupled Rotation Mechanismcited by 121
  29. 2021SE-SSD: Self-Ensembling Single-Stage Object Detector From Point Cloudcited by 119
  30. 2021RSN: Range Sparse Net for Efficient, Accurate LiDAR 3D Object Detectioncited by 115
  31. 2023Benchmarking Robustness of 3D Object Detection to Common Corruptions in Autonomous Drivingcited by 112
  32. 2021Improving 3D Object Detection with Channel-wise Transformercited by 108
  33. 2021Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentationcited by 107
  34. 20213D Object Detection for Autonomous Driving: A Surveycited by 106
  35. 2021DualPoseNet: Category-level 6D Object Pose and Size Estimation Using Dual Pose Network with Refined Learning of Pose Consistencycited by 103
  36. 2022Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Cloudscited by 102
  37. 2022Semantic Abstraction: Open-World 3D Scene Understanding from 2D Vision-Language Modelscited by 95
  38. 2021SPG: Unsupervised Domain Adaptation for 3D Object Detection via Semantic Point Generationcited by 93

50 most cited of 640 All 640 (JSON)