Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images
Not on the map. Why?
Abstract (arXiv)
We focus on the task of amodal 3D object detection in RGB-D images, which aims to produce a 3D bounding box of an object in metric form at its full extent. We introduce Deep Sliding Shapes, a 3D ConvNet formulation that takes a 3D volumetric scene from a RGB-D image as input and outputs 3D object bounding boxes. In our approach, we propose the first 3D Region Proposal Network (RPN) to learn objectness from geometric shapes and the first joint Object Recognition Network (ORN) to extract geometric features in 3D and color features in 2D. In particular, we handle objects of various sizes by training an amodal RPN at two different scales and an ORN to regress 3D bounding boxes. Experiments show that our algorithm outperforms the state-of-the-art by 13.8 in mAP and is 200x faster than the original Sliding Shapes. All source code and pre-trained models will be available at GitHub.
Key facts. Cited by 111 papers in the map (data through 14 Oct 2026). First posted on arXiv 7 Nov 2015.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 41; 2022: 43; 2023: 13; 2024: 7; 2025: 5; 2026: 2.
111 papers.
Sort:
By area:
- 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329
- 2021PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detectioncited by 232
- 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
- 2021Group-Free 3D Object Detection via Transformerscited by 215
- 2021Offboard 3D Object Detection from Point Cloud Sequencescited by 145
- 2021Revisiting Point Cloud Shape Classification with a Simple and Effective Baselinecited by 145
- 2021FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimationcited by 144
- 2021LiDAR R-CNN: An Efficient and Universal 3D Object Detectorcited by 129
- 2021SAT: 2D Semantics Assisted Training for 3D Visual Groundingcited by 124
- 2021Improving 3D Object Detection with Channel-wise Transformercited by 108
- 20213D Object Detection for Autonomous Driving: A Surveycited by 106
- 2022Semantic Abstraction: Open-World 3D Scene Understanding from 2D Vision-Language Modelscited by 95
38 more
- 2022Surface Representation for Point Cloudscited by 84
- 20213D Semantic Scene Completion: a Surveycited by 74
- 2021Category-Level 6D Object Pose Estimation via Cascaded Relation and Recurrent Reconstruction Networkscited by 69
- 2021Back-tracing Representative Points for Voting-based 3D Object Detection in Point Cloudscited by 67
- 2023Deep Learning on Implicit Neural Representations of Shapescited by 63
- 2021Pri3D: Can 3D Priors Help 2D Representation Learning?cited by 61
- 2022Motion Inspired Unsupervised Perception and Prediction in Autonomous Drivingcited by 59
- 2021Semantic Scene Completion via Integrating Instances and Scene in-the-Loopcited by 55
- 20214DContrast: Contrastive Learning with Dynamic Correspondences for 3D Scene Understandingcited by 54
- 2022Polar Parametrization for Vision-based Surround-View 3D Detectioncited by 53
- 2023PiMAE: Point Cloud and Image Interactive Masked Autoencoders for 3D Object Detectioncited by 53
- 2022SASA: Semantics-Augmented Set Abstraction for Point-based 3D Object Detectioncited by 52
- 2021EPNet++: Cascade Bi-directional Fusion for Multi-Modal 3D Object Detectioncited by 49
- 2022NeRF-RPN: A general framework for object detection in NeRFscited by 46
- 2021Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Drivingcited by 45
- 2022Look Around and Refer: 2D Synthetic Semantics Knowledge Distillation for 3D Visual Groundingcited by 41
- 2022RBGNet: Ray-based Grouping for 3D Object Detectioncited by 40
- 2021VoRTX: Volumetric 3D Reconstruction With Transformers for Voxelwise View Selection and Fusioncited by 38
- 2021RandomRooms: Unsupervised Pre-training from Synthetic Shapes and Randomized Layouts for 3D Object Detectioncited by 37
- 2023Unsupervised 3D Perception with 2D Vision-Language Distillation for Autonomous Drivingcited by 37
- 20213D Spatial Recognition without Spatially Labeled 3Dcited by 31
- 2022Guaranteed Conservation of Momentum for Learning Particle-based Fluid Dynamicscited by 30
- 2021PQ-Transformer: Jointly Parsing 3D Objects and Layouts from Point Cloudscited by 26
- 2022Weakly Aligned Feature Fusion for Multimodal Object Detectioncited by 26
- 2024A Survey on Text-guided 3D Visual Grounding: Elements, Recent Advances, and Future Directionscited by 25
- 2022A Closer Look at Invariances in Self-supervised Pre-training for 3D Visioncited by 24
- 2021ODAM: Object Detection, Association, and Mapping using Posed RGB Videocited by 22
- 2022Uni6D: A Unified CNN Framework without Projection Breakdown for 6D Pose Estimationcited by 22
- 2022LCPFormer: Towards Effective 3D Point Cloud Analysis via Local Context Propagation in Transformerscited by 21
- 2025SuperPC: A Single Diffusion Model for Point Cloud Completion, Upsampling, Denoising, and Colorizationcited by 21
- 2021MonoGRNet: A General Framework for Monocular 3D Object Detectioncited by 20
- 2022Bridged Transformer for Vision and Point Cloud 3D Object Detectioncited by 20
- 2021Monte Carlo Scene Search for 3D Scene Understandingcited by 19
- 2021NeRD: Neural 3D Reflection Symmetry Detectorcited by 18
- 2021Revisiting 3D Object Detection From an Egocentric Perspectivecited by 18
- 2022DArch: Dental Arch Prior-assisted 3D Tooth Instance Segmentationcited by 18
- 2022Back to Reality: Weakly-supervised 3D Object Detection with Shape-guided Label Enhancementcited by 17
- 2022PointDP: Diffusion-driven Purification against Adversarial Attacks on 3D Point Cloud Recognitioncited by 16
50 most cited of 111 All 111 (JSON)
- 2026A Cookbook of 3D Vision: Data, Learning Paradigms, and Application2 Jun 2026
- 2026UniD-Shift: Towards Unified Semantic Segmentation via Interpretable Share-Private Multimodal Decomposition8 May 2026
- 2025SDT-6D: Fully Sparse Depth-Transformer for Staged End-to-End 6D Pose Estimation in Industrial Multi-View Bin Picking9 Dec 2025
- 2025GraphFusion3D: Dynamic Graph Attention Convolution with Adaptive Cross-Modal Transformer for 3D Object Detection2 Dec 2025
- 2025Investigating Domain Gaps for Indoor 3D Object Detection24 Aug 2025
- 2025Gaussian2Scene: 3D Scene Representation Learning via Self-supervised Learning with 3D Gaussian Splatting10 Jun 2025
- 2025SuperPC: A Single Diffusion Model for Point Cloud Completion, Upsampling, Denoising, and Colorization18 Mar 2025
- 2024Gaussian-Det: Learning Closed-Surface Gaussians for 3D Object Detection2 Oct 2024
- 2024General Geometry-aware Weakly Supervised 3D Object Detection18 Jul 2024
- 2024CT3D++: Improving 3D Object Detection with Keypoint-induced Channel-wise Transformer12 Jun 2024
- 2024A Survey on Text-guided 3D Visual Grounding: Elements, Recent Advances, and Future Directions9 Jun 2024
- 2024TexVocab: Texture Vocabulary-conditioned Human Avatars31 Mar 2024
38 more
- 2024A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes12 Mar 2024
- 2024Atlanta Scaled layouts from non-central panoramas30 Jan 2024
- 2023Deep learning for 3D Object Detection and Tracking in Autonomous Driving: A Brief Survey10 Nov 2023
- 2023Anyview: Generalizable Indoor 3D Object Detection with Variable Frames9 Oct 2023
- 2023ShaSTA-Fuse: Camera-LiDAR Sensor Fusion to Model Shape and Spatio-Temporal Affinities for 3D Multi-Object Tracking4 Oct 2023
- 2023Unsupervised 3D Perception with 2D Vision-Language Distillation for Autonomous Driving25 Sep 2023
- 2023A Fusion of Variational Distribution Priors and Saliency Map Replay for Continual 3D Reconstruction17 Aug 2023
- 2023MoDAR: Using Motion Forecasting for 3D Object Detection in Point Cloud Sequences5 Jun 2023
- 2023SG-GAN: Fine Stereoscopic-Aware Generation for 3D Brain Point Cloud Up-sampling from a Single Image22 May 2023
- 2023PiMAE: Point Cloud and Image Interactive Masked Autoencoders for 3D Object Detection14 Mar 2023
- 2023Surface-biased Multi-Level Context 3D Object Detection13 Feb 2023
- 2023Deep Learning on Implicit Neural Representations of Shapes10 Feb 2023
- 2023Joint Representation Learning for Text and 3D Point Cloud18 Jan 2023
- 2023Object Detection in 3D Point Clouds via Local Correlation-Aware Point Embedding11 Jan 2023
- 2023Rethinking Voxelization and Classification for 3D Object Detection10 Jan 2023
- 2022MM-3DScene: 3D Scene Understanding by Customizing Masked Modeling with Informative-Preserved Reconstruction and Self-Distilled Consistency20 Dec 2022
- 2022Context-aware 6D Pose Estimation of Known Objects using RGB-D data11 Dec 2022
- 2022Review on 6D Object Pose Estimation with the focus on Indoor Scene Understanding4 Dec 2022
- 2022Look Around and Refer: 2D Synthetic Semantics Knowledge Distillation for 3D Visual Grounding25 Nov 2022
- 2022NeRF-RPN: A general framework for object detection in NeRFs21 Nov 2022
- 2022ImLiDAR: Cross-Sensor Dynamic Message Propagation Network for 3D Object Detection17 Nov 2022
- 2022Recursive Cross-View: Use Only 2D Detectors to Achieve 3D Object Detection without 3D Annotations14 Nov 2022
- 2022PointSee: Image Enhances Point Cloud3 Nov 2022
- 2022Seg&Struct: The Interplay Between Part Segmentation and Structure Inference for 3D Shape Parsing1 Nov 2022
- 2022Li3DeTr: A LiDAR based 3D Detection Transformer27 Oct 2022
- 2022LCPFormer: Towards Effective 3D Point Cloud Analysis via Local Context Propagation in Transformers23 Oct 2022
- 2022Learning Point-Language Hierarchical Alignment for 3D Visual Grounding22 Oct 2022
- 2022Class-Level Confidence Based 3D Semi-Supervised Learning18 Oct 2022
- 2022Motion Inspired Unsupervised Perception and Prediction in Autonomous Driving14 Oct 2022
- 2022Guaranteed Conservation of Momentum for Learning Particle-based Fluid Dynamics12 Oct 2022
- 2022Prototypical VoteNet for Few-Shot 3D Point Cloud Object Detection11 Oct 2022
- 2022Bridged Transformer for Vision and Point Cloud 3D Object Detection4 Oct 2022
- 2022FusionRCNN: LiDAR-Camera Fusion for Two-stage 3D Object Detection22 Sep 2022
- 20223DLG-Detector: 3D Object Detection via Simultaneous Local-Global Feature Learning31 Aug 2022
- 2022PointDP: Diffusion-driven Purification against Adversarial Attacks on 3D Point Cloud Recognition21 Aug 2022
- 2022Semantic Abstraction: Open-World 3D Scene Understanding from 2D Vision-Language Models23 Jul 2022
- 2022Boosting 3D Object Detection via Object-Focused Image Fusion21 Jul 2022
- 2022A Closer Look at Invariances in Self-supervised Pre-training for 3D Vision11 Jul 2022
50 newest of 111 All 111 (JSON)
- 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329
- 2021PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detectioncited by 232
- 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
- 2021Group-Free 3D Object Detection via Transformerscited by 215
- 2021Offboard 3D Object Detection from Point Cloud Sequencescited by 145
- 2021Revisiting Point Cloud Shape Classification with a Simple and Effective Baselinecited by 145
- 2021FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimationcited by 144
- 2021LiDAR R-CNN: An Efficient and Universal 3D Object Detectorcited by 129
- 2021SAT: 2D Semantics Assisted Training for 3D Visual Groundingcited by 124
- 2021Improving 3D Object Detection with Channel-wise Transformercited by 108
- 20213D Object Detection for Autonomous Driving: A Surveycited by 106
- 2022Surface Representation for Point Cloudscited by 84
12 most cited of 105 All 105 (JSON)
- 2026A Cookbook of 3D Vision: Data, Learning Paradigms, and Application2 Jun 2026
- 2026UniD-Shift: Towards Unified Semantic Segmentation via Interpretable Share-Private Multimodal Decomposition8 May 2026
- 2025SDT-6D: Fully Sparse Depth-Transformer for Staged End-to-End 6D Pose Estimation in Industrial Multi-View Bin Picking9 Dec 2025
- 2025GraphFusion3D: Dynamic Graph Attention Convolution with Adaptive Cross-Modal Transformer for 3D Object Detection2 Dec 2025
- 2025Investigating Domain Gaps for Indoor 3D Object Detection24 Aug 2025
- 2025Gaussian2Scene: 3D Scene Representation Learning via Self-supervised Learning with 3D Gaussian Splatting10 Jun 2025
- 2024Gaussian-Det: Learning Closed-Surface Gaussians for 3D Object Detection2 Oct 2024
- 2024General Geometry-aware Weakly Supervised 3D Object Detection18 Jul 2024
- 2024CT3D++: Improving 3D Object Detection with Keypoint-induced Channel-wise Transformer12 Jun 2024
- 2024A Survey on Text-guided 3D Visual Grounding: Elements, Recent Advances, and Future Directions9 Jun 2024
- 2024TexVocab: Texture Vocabulary-conditioned Human Avatars31 Mar 2024
- 2024A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes12 Mar 2024
12 newest of 105 All 105 (JSON)